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ABSTEACT ' ' - . 

^ . ' ^ This is the final xeport of one of th-ree studies in 

an.,ovei:ali project entitled "Evaluation of Bilingual Education. ' • 
Prograas." Tlie objectives of this stud.y were to: <T) identify and 
recoQBen<J ^s. exeaplary up to 10 jailingual education prograiis for 
subaission*''to tie Dissenination Review- Panel (DSP) of the Education 
Division^ D5EW,^and (2) develop derailed descriptions of those 
prograns 'approve3* by the DBP for disseMination. Screenings and site 
yisLts -rediiced -the candidate^pbol of 175 programs to 7> The DEP 
approved the following • four for disseaination: A£Lice Independeiit 
School Distric£v Alide, .Texas; ApretfdeaBs en Dos Tdioaas, Corpus 
Cbt'isti, Texas; Bilingual Education Program, Houston, Texas; and St. 
John Valley Bilingual Education Proj^ani, Madavaska, Maine. 
Descriptions were written for each of the four in sufficient detail 
to provide* local educatprs .wirh* ideas to guide thea in iapleaent:^ng 
similar practices. Each .program was described, as 'it operate^ during 
1973-74/ but sbae^ program coiliponeitts wera* treated historically .to 
illust:r:ate major changes.. The context withih which each program- 
developed and .offe rated, and" the special educational needs of^the 
students, were also discussed. In addition, specific recommendations 
were aade oA.ways in which local evaluations could be improved' to- 
increase chances for identifying exemplary prograas. (Author^^EM) 
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PREFACE 



This document is the finalj^eport of one of three studies conducted by the 
American Institutes for Resear^ch (AIR) for the Office of Education, Office 
of Planning; Budgeting, and Evaluation, under Contract No. OEC-0-74-9331. 
The OE'Project Of f icer Vas Edward S.' Glassman. The study reported in this 
document was co-directed by Peggie Campeau and John Bowers, and the AIR 
project directpr for the overall contract was Malcolm N. Danoff . 

The study reported here is entitled "The Identification and Description of 
Exemplary Bilingukl Education Programs.". Its objectives were to recommend 
"to the Joint Dissemination Review Panel (DRP) of the Education Division. . 
. Department of Health. Education, and Welfare, .those bilingual education r 
programs with sound: evidenc.- demonstrating significantly iiR,rov«i- student 
outcomes, -and to prepare da't^led dwpriptiotis fotJat^e-jrogram.- th.fi. tU:?)RF 
approved for dissemination. 
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SUMMARY ^ ~ * 

? ^ * ■ 

\ - ^ ~ , 

The objectives of this study,' conducted by the American institutes for 
Research (AIR) under contract to the U.S; Office of Education, were (1) to 
identify and to recommend as exemplary up to 10 bilingual education programs 
-for submission "to the Dissemination Review Panel (DRP) of the Education 
Division, Department of Health, "Education , and Welfare, and (2) to develop 



detailed descriptions of those pr ograms appro ^red by the DRP for dissemination. 

The Officfe of- Planning, Budgeting, and Evaluation (OPBE) and AIR coopera- 
tively defined criteria' for screening candidate programs. The criterln were 
bas^upon several requisites. To be considered, a candidate program mt.st 
incltide English language instruction for children witH limited EngliTsh skills ^ 
and must teach in the.^ujpils' ngti«e lang^age to the -extent necessary to allow 
,^hem to progress effectively through school. Part of the curriculum must also Be 
addressed to the customs and cultural History of the native language group. 
^FClrthennqre,' program participants must show statistically and. educationally 
Significant gains in English language skills as well as in the "content areas, 
taught in the. native language.- The program must alsp have definable and' ^ 
describable instructional and management components, finally, start-up 'and 
continuation costs must be reasonable. . " ' 

Candidate programs wer4 located from a variety o'f soiwrces^. Initial loads . 
came from' the Offi<:e of Education and from Title VII files, which were, survi-vcd 
by the project director at the beginning. of the' study! ' Other sources were . 
programs, surveyed for past AIR studies , 'programs' reviewed in a previous study 
bvthe RMC Research Corporation, regional educational laboratories, state 
bilingual education officials, and local education agencies., There werd 
175 programs in the total candidate pool. " 
J -■ A telephone screening was conducted, to determine whether programs met 
minimum requirements to be "considered further, and to request that programs 
send- evaluation reports. As a result of the telephone ^screening; 59 programs 
were dropped.. Ap additional 20 programs that had been telephoned could nd't 
be considered beoausW" evaluation reports were not received for them. 

The remaining' 96 candidates were screened by at least two AIR senior 
staff' members. Information fbr each program, summarized on a rating form, . 
was the basis for a preliminary judgment "about the program. The rating form 
'included .sections concerned with the measures, sampling, grade levels , -statls- 
tics used, and the. significance of -program. effect, 

. ' ■ 12 • ' 



■ Nineteen programs met the screeniftg -criteria sufficiently to warrant" 
further- consideration; 'll of these programs wer^ subsequentlydropped. Eipht 3 

. programs survived both the initial screening procedure and^tbe foilowing deiniled 
examination and were selected for site visiting. 

The^wrposf of site ivisiting was to verify evdBence of the program'? 
effectiveness as reported in evaluation reports and to observe how the pro- 
gram operated. One program was dropped from 'the study af.ter the_ si te visit. 

"ihe remaining' seven f>rograms were, recommended to the DRP by AIR a^i exetnpl.irv 
bllingu^r'edaeation programs. S'ummariefi_ of program operations -ind evaluation • _ 
data for "the seven were prepared and submitted to .-the DRE for review. The 
DRP approved four "of. the seven for dissemination, as follows: . 

J..' Alice. Independent Schaol District 
Bilingual Education ^rbgram 
* • Alice-^ Texas • . • • . — 

, • ■ 2. Aprendeiaos en Dos'Idiomas _ . ' 

' Title VII Bilingual Project . • ... ' 

Corpus Christi, Texas • - ' : . 

\ ■ - ' . 

3. Bilingual Education Program • ' , 

» „ Houston, Texas 

4. St. 'John Valley. Bilingual' 

Education Program • ^ " * 

Mad awaska,* Maine • ' * . • . 

Descriptions wete written for these four ex-emplary programs in sufficient 
detail to provide local educators with a source of ideas to " guide _ them In im- ( 
plemen'ti^ig similar bilingual education practices in their districts. Each 
program was described as 'it operated during the year for which the most tecent 
■ 'and complete evaluation information wa8_ available (in all cases, 1973-74). 
•Some ptogram components were treated historically.. to .illusOrate major changes 
.that had.occurred since the program began. The descriptions also discussed 
the context within which e4ch prggram developed and operated, and the special 
• educational ne^s of the students. 

a. rifsult pf the study, AIR made specific recommendations to school 
i districts and pro'grai evaluators on ways in which local evaluations couW lyfe^ 
^ improved to increase the chances for identifying additional exemplary programs 
J.n bilingual education. ^ 
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CHAPTER I / , 

INTRODUCTION ^ ''^^ . 

« 

• ♦ 



This-document is the final report of one of three studies conducted by 
the American Institutes for Research (AIR) id the area^of bilingual education 
under contract to* the uVs* Office of Education. The study reported here aimed 
to identify .and describe successful bilingual education programs in American 
schools* It is intended that programs described wilL serve as models for 
school systems seeking to improve the quality of the education" they .of fer" to 
Children who enter school with little or no ability to communicate in English. 

The major tasks o^ the study were to: * " - . 

r cailabotate, with the Office of Education in refining criteria 
for selecting exemplary, bilingual education programs, 

• iearch for programs which met these cpitjeria, 

• develop" procedures to review ahd to screen programs 
according to' the criteria, ' . 

• visit selected programs to obtain- further evaluative and 
descriptive information, / ^ ^ 

• submit to the Office of Educatioji and to the Dissemination 
Review Panel (DRP) of the 'Education .Division,. Department of ' 
Health, Education, and Velfade, sumnaries of bilingual programs ^* , 
recommended by AIR, and ' . * ^ 

• prepare detailed descriptiotis of the bilingual programs approved 
for dissemination by the DRP. 

' The next six'chapters of this report- correspond to these six major^task«. 
Chapter II includes a list^g, defi-nition, Ind^discussion oi criteria used to^ 
identify exemplary bilingual programs. Chapter ^11 details ^the procedures for^ 
locating candidate programs and , acquiring data and documents needed to screen''- 
programs according -t6 the exemplary criteria. Chapter IV describes the revie^w*-. 
and-screep procedures for applying criteria to identify ^the mdst promising 
programs for site visits? Chapter V, summarises the purposes, |)rocedures, and 
results of the sit^ visits. Chapter VI pi;esents procedures for recomnending ^ 
exemplary programs for dissemination and summarizes- decisions by the DRP on 
■•these recommendations. Chapter VII describes- the preparation -of detailed' 



descriptions for programs approved for dissemination by the m?. The final 
chapter .is a discussion of the results of the study and recommendations for 
Improving local program evaluations- 

•1 Limits of the Study 

• A major 3?imitation of this study was its dependence on available, recent 
evaluation information. Programs that had not conducted evaluations or that 
did not supply summaries of these dat^a i'o were not even initially consider- 
ed. All conclusions reached in this study, then, apply only to thdse* initially 
identified programs that were contacted by AIR and which supplied program docu- 
ment^ to AIR. 

^second major limitation was the study's dependence on the 'review of 
existing program e^valuations • It was not possitrle to do extensive reanalysis 
of raw data, nor was it possible to compensate for poorly pl^ed local evalba- 
tions. Telephone interviews and, in the case of visited programs, site inter-- 
views, had to provide needed clarifications t 

The sources from which leads to candidate, prdograms were obtained also 
limited *the scope of'Clji's study. The original intent was that nominations 
be provided by the Office of Education. Sources of possible exemplary pro- 
grams were maittly lists .supplied by Office of Education and included pro- - 
grams funded under various titles of the Elementary and Secondary Education Act 
(ESEA)' and by the Emergency School Assistance Act (ESAA) . As specified in the 
Request for Proposals, bilingual programs, for the handicapped were excluded, 
regardless of funding source. 

^ A fourth Important study limitation was set by the criteria for screening' 

""programs. These criteria, established by the Office of EdjiC^tion and refined 
in collaboration with AIR,, intentionally limited the choice of programs to those 
in which program treatment was consistent with Title VII legislation and defini- 
tions, and for 'which evidence of effectiveness was statistically treated, based 
on measured achievement and comparison between program participants and some 
suitable baseline. An example of a disqualification for the former reason 
would be a remedial reading program for Spanish-surnamed , English-speaking 
participants who did not need bilingual instruction in order to progress in 
school. An e:^ample of a disqualification for the latter reason would be a 
bilingual progr-am with evidence of effectiveness that could not justifiably 

.be attributed to the impact of the bilingual program because no baseline or 



cooparison' could be provided -against* which to judge gains made by the partici- 
pants. In Chapters 11 and IV in this report, these -and other criteria are 
defined and the t'esults &f their application in ^creening programs are discussed 

Finally,, the products of this study-detailed descriptions of bilingual 
education pr;ograms~were limited to programs approved as exemplary by the DRP. 
The main objective of the DRP^ review was to scrutinize again the evidence of 
effectiveness presented for a prpgrdm and to decide if outcomes were of .suffi- 
cient importance* to make the program worthy of endorsement by the federal govern 
ment-for use by others. 



CHAPTER II ^' ' 

,BAS1S OF THE CRITERIA JOR EXEMPLARY STATtfS r * * 
* «. ■ 

General Consider a tlong » . 

Exemplary programs in bilingual education are j^dentifled in ord^r that 
information about them may be disseminated for replication. They are programs, 
that show unusual merit, both in terjns of^ greater than ordinary cognitive ai^d 
affective gains for their pupils, as well as in the quali.tj of their program 
el-emepts or components. 

Programs 'judged to be exemplary were recommended by AIR for review by Che 
Dissemination Review Panel (DRP) of the Education Division, Department of Health, 
Education, and Welfare, which approves t^e disseraination of • information' about 
effective educational practices, products, and projects funded with federal 
monies, fn'effect, the DRP issues a. federal stamp of approval, labelling as * 
exemplary those educational pv^gvame recommended to it that are considered by 
the DRP to be worthy of replication. Based upon review of program documents, 
consultation with program staff, and site visits, program descriptions are pre- 
pared and submitted to the DRP for review and judgment. 

Two major difficulties are encountered in searches for exemplary programs. 
First, relatively few local program evaluations organize data in a way that 
permits the results for program pupils to be compared with the results that 
might be expected otherwise'. Second, even when such comparisons are attempted, 
"relatively fewer local program evaluations construct proper design safeguards 
agairtst alternative interpretations of program outcomes. Consequently, the 
application of **unusual merit" criteria with respect to cognitive or affective 
gains /-esults ip the identificat^ion of only a handful of programs .that can be 
labelled' exemplary. 

The first task in any searpK^f or (exemplary programs is tp define 
criteria that can be agpli-ea in order to classify those labelled exemplary 
and tho9JB,^not'r''TlTfe8e criteria serve as, the basis for program 'evaluation. 
^Since programs are topically described in terms of their components, it is 
ideally possible to d^elop. evaluative criteria for each of thesg program 
el^ents. Also, these criteria must relate to the objective of replication. 
Aside from the fact that it is strategically mora sound to copy a program 
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with greater than average pupil outcomes rather thin orie showing ordinary 
results, replicability potential jaust also -be judged .on the basis of other.. ^. 
important program features, including reasonable costs, clear instructional 
processes, and .^ear management procedure's. However, with the exception of 
'general criteria relating to reasonable costs and the clarity with which in- 
structional and management proqesffes can be described; consensus regarding 
appropriate criteria to evaluate other program processes is rare. Typically, 
such criteria applied, to program^ processes are -those developed and applied 
during formative eva],-uation. . • ' ' 

Evaluation.' studies designed to locate exernplar>' programs .favor summatlve 
rather than formative evaluation, principally because it is assumed that mpt 
developmental ^problems have been successfully solved as the program matured. 
ThJs most program processes are dfescrlbed but not ^evaluated; instead, the pri- . 
mary evaluation focus- is upon'iocal comijarispn studies shqwing' that pupils 
enrolled- in the program demonstrate significantly better achievement than 
would otherwise occur through tfheir participation in a regular or alternative 
treatm<iit.' An e^lary program is one that produces extraordinary outcome*. 

Em'phasiB on this criterion presents' a:dileBma for the program developer^, 
especially one who attempts replication/ The problem is that altljough program 
benefits are clear, the relationship between outcomes and program processes is 
. uncertain. When attempts to replicate are unsuccessful, one is unsure how' to 
adapt. the replicated program in order to produce the beneficiai outcomes obtained 
at the original program site. ' . ^ 

In the absence of known program process-program outcome relationships which 
permit adaptations to be made 'in replication attempts, it is necessary to devel- 
op ancillary criteria that focus on the generalizabllity and? hence, the replic*- 
- bility potential of the programs.. These criteria involve.^ucb factors as "the 
program's durati.on, its size, its benefits across classes dnd grades, the 
appropriateness of iteasures used to assess its pupils' achievement, its expense, 
and the clarity with which its instructional and management processes and objec- 
tives can be described and understood. Thus., the demonstrated positive effects 
of an exemplary program upon its pupils must be impressive, and the evidence, of 
■ effectiveness must be sufficiently generalifcble to permit reasonable confidence 
in the program's potential for replicability. ^ , 
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Legislative Implications 
While .the above 'considerations apply in general to all edutatTonal programs, 
the search for exemplary bilingual education programs must also concentrati^von 
the unique characteristics of bilingual education as defined in federal legisL 
tion. Bilingual education seeks to change existing more traditional instruction- 
al systems in major ways. Its philosophy calls for early instruction in a 
dominant language other than English. This is an extr^ innovation whose effects 
are transmitted- to all parts of any. bilingual 'education program, including needs 
assessments, teacher preparation, the selection and development of materials, 
parental and community involvenent, and certainly, the formative proces* evalua- 
tion of the program. Ideally, all of these elements would be assessed in a 
program evaluation. 

Federal legislation for bilingual education indicates three key featyres. 
First, instruction in the English language is to be provided and Engll^sh is to 
be studied. Second, to the extent necessary tp allow a child to progress 
effectively through the educational system, instruction in a child^s native 
language must be provided to children of limited English-speaking ability. 
Furthermore, instruction in the child's native language is to be provided with 
• appreciation for his cultural heritage, and in all courses or subjects of study 
which will allow his effective progress throygh the educational system. Third,, 
a bilingual program must demonstrate effective ways of providing instruction to 
children of limited English-speaking ability to enable them, while using tl}e^r 
native language, to achieve competence in theoEnglish language. 

Programs that stress only the teaching of English as a second language, or 
tfhe teaching of a second^ language to English-speaking children who may volun- 
tarily enroll in the bilingual prbgram do not qualify within the legislative 
definition of a bilingual program. . ^ ^ ^ 

Criteria for Exemplary Bilingual Education Proferams ^ 
As iridicated above, a .bilingual program is- one whose participants are 
instructed both in English and in their native, language to l^e extent netessary 
to ensure their effective educational progress. Thu^ criteria used to identify 
exemplary bilingual p/ograms must concentrate on the development of English 
language skills as'well as the development of necessary progress-related native 
' language skills. Measures relating to both positive self-concept and to the 
development of the appreciation of the participants' cultural herit/ge are 
important, but §re not «tif«.cient by themselves ^to id^gntify a bilingual program 

f 
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Outcomes ^ * " . 

EvalCaeionfl of toceoplary bilingual programs must therefore show the follew- • 

irtg'outtsaea: 

(1) Children of limited English-speaking ability achieve edudationally 
and statistically significant gains in English-language skills in comparison 
with similar pupils in regular or alternative programs. Comparison results are 
typically based on program-control ^group gain contrasts, or show accelerated 
normatiye growth over time, or demonstrate improyed outcomes for pupils in the 
programs in contrast to outcomes observed for similar pupils enrolled in the 
•same grade levels prior to. the introduce ion* of the program, pleasures, of these 
English- language gains will customarily be based upon the administration of 
standardized tests. - 

(2) Children of limited English-speaking ability achieve both educationally' 
relevant and statistically signif ican&.gains in their native-language skills 
when instruction in their native language is provided "to the ej^tent necessary 

to- ensure their effective educational ptogress. Evidence, of effectiv.e prorgress 
must be demonstrated, and, since common normative measures of content achievement 
'are usually unavailabJLe, acceptable 'evidence may include teachers' grades" or 
judgments of pupil progress. Grades are rarely reported in summative evaluations. 

(3) For children whose dominant language iff English and who voluntarily 
enroll in a. bilingual program, reported evidence shows that their development 
of English-language skills equals that expected if they were instead enrolled 
in a regular educational, program. Acceptable evidence includes test scores, 
teachers' grades, or teachers' opinions. 

Acceptable Evidence 

Evidence of bilingual program impact should be based on objective measure- 
ments obtained from sizeable pupil s^ples. Achievement gain measures should 
be estimated for program -participants and for a comparable control group. Well-^ 
designed contrasts with pre-program baseline or comparison with appropriate 
norm reference groups are alsd acceptable. It is necessary that gains for 
program participants be significantly greater than gains ^f or the control or 
comparison group. 

Interpretation of the significance of the reported gains depends on 
customary psychometric and statistical-grounds. Measurements should be rcli.able 
and valid. Tests should be of appropriate difficulty level for the groups 
examined. The reporting of achievement in either grade-equivalent or raw-score 
scales is acceptable; one scale is essentially a linear transformation of the 
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relative withJ.n-group stan- 
ces of the order of one-halT 



other except' at extreoe ranges." Average gaLs for pupils in the caparison 
groups siioald "be unbiased estimates of the gains for the total population of - 
participants; that is, missing data or the effects of selection should not "be 
great enough to cast doubt on the findings, donfid^ace in the generalizabiliCy 
and potential for replicability are also great^tr when results are reported for 
several- classes and grade levels, so that uni^t^^^cher or administrator effects 
can be ruled out. 

Statistical significance should be demonstrVted so that one may confidently 
ionclude that the results showing superior prdgrAn effect did not occur by^ 
chance; that is, 'results showing significant program effect, when In fact there 
is none, should occur no more than five perceiit of the time. In addlticfn, mean 
gain differences between program and control groups must be educationally rele- 
vant whether reported as grade equivalents or as 
dard deviation units. For example, mean dif,fererj< 
grade equivalent, or one-half standard deviation, are meaningful, as is a large 
positive shift, in mean perqentil.es between pre- and posttests whefi program out- 
comes a're compared to those of norm reference groups. _ , 

The number 0^ •yea'^fin which the pjogram has been in existence a« well 
as its expected S)eri<od of contintiation are important program characteristics. 
• A pr'ogram which has been in existence for several years -»tjggests not only that 
formative evaluations leading'.to process improv^ts have taken place, t.ut 
also that 'the critical change from start-up to maintenance management has 
occurred. Expected longevity suggests confidence in the local district's 
acceptance of the program and' equally important, indicates that the original 
ongoing program cah operate as a resource model for program planners attempt- 
ing its replication. 

Clarity of Instructional and Management Procedures 
and Availability of Materials 
To qualify as an exemplary bilingual program," that is, one whose 
potential for jfeplicability is strong, it is essential that instructional ^ . 
and management processes be ^ell-defined. Program dbiponents- and^^concepts ' 
should be presented in a coherent and^analyzable fashion. structured- to 
assist competent administrators and teachers who attempt its whole replica- 
tion *br who wish to modify, existing programs. Instructional materials must 
be available for dissemination to bilingual educators. The implication here 
• is that -copyrights, ^patents, and othe^ jropri^tary*arrangements should not" 
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impede the full dissemination these materials. 

. ' Reasonable Start-Up and Recurr ing Co.sts 

program costs are essential items of inforiiation for persons planning 
replication. While it is recognized that' variability in total costs and 
in cost categories exist aootiJg programs and locations, it is. nevertheless^ ^ 
important to provide accurate start-up and recurring budget guidelines iSx 
administrative, clerical, teacher and teacher-aide salaries, materials, 
equipment, facilities, consultants, services, travel, testing, and evalua- 
tion. Total per-pupil costs must be within acceptable district limits 
for replicability. and must be presented in such a way that any add-on 
costs for bilingual education are clear. 

Final Considerations > 
The decision to recommend a bilingual educational program as exemplary 
is clearly a summative judgment, arrived at through application of the 
criteria discussed in the preceding sections of this chapter. Few programs 
can be expected to show exceptional strengthson all criteria; thus the., 
final decision is based upon a weighing -of strengths and weaknesses. Al- 
. though all programs ^mmended as exemplary must demonstrate significant , 
achievement gains foftheir participants, it is impqrtant to understand -that 
program components other .than merely the. program evaluation design are 
'also considered in arriving at a final judgment. Th^se are. as has be.en^ 
discussed, cle-ar instructional and management procedures, availability 
of materials, and reasonable start-up and recurring costs. T4iese all are 
■ essential considerations when j-udging a program"'? potential for replicabil- 

lty~which is the crucial reason for identifying exemplary proRram& in 
^ the first place. 
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' CHAPTER III 

IDENTIFYING CANDIDATE .PROGRAMS FOR THE STUDY 

■ 

Sources of Leads to Ca ndidate PVograms 
Leads to candidate program^ for consideration in this study were supplied 
by the Office of Education. It was originally believed that up to^lO exem- 
plpry bilingual programs could be identified from this pool. Howdyfer. addi- 
tional sources of leads to candidate programs were explored. These addi- - 
tional sources included AIR's extensive 'program files from a natibn-vide search 
for exemplary reading programs for the National Right to Read Office, the 
KMC Research Corporation's files from a., oh-frmdcd sear, h for exem,.Jarv 
programs in reading and math for disadvantaged children., and nominations 
obtained in the course of seekii infcmnatien on current study candidates from 
regional educational laboratories, state bilingual education officials, and 

local educational agencies. 

- The candidate programs identified through these sources included programs 
funded by Title VII. Title III. Title IV. Title I, and the Emergency School 
Assistance Act (ESAA) . Contacts with Title VII programs were generally limited 
to those tha£ had been operating at least four years. This period of time was, 
judged to be-^ufficient for a program .to- have collected, analyzed, and sum- 
marized evaluation data for at least two or three academic years. Also, pro- 
grams In operation this long were reascftied to be relatively stable. Leads 
supplied by sources other than Title VII were followed, dp regardless of how 
long the programs had been operating, because these nominations had been pre- 
salected by officials who were aware of the objectives of^ the study. 



:ontacting Candidate Programs 
A letter from the ctmissioner of Education was' maile4 to ever/ Chief 

ore programs were contacted by k^^. A copy of the 
_ contained in Appendix B. The letter informed state 

education officials aboit the objectives of the study and indicated that con- 



State School Officer bei 
'Connnissioner^s letter i* 



tacts with local progran 



staff would be made in order %o obtain information 

about programs considered for the study. 

staff then telephoned project ' directors or other cognizant local, 
education officials to inquire about" each of the candidate programs'. A total 




of 14A programs were .called. During these telephone conversations, AIR staff 
explained the purpose of the study and hoW the programs being contacted had 
been identified. The local- contact* werej^riefed on the need for evidence 
of program ef f ectiveneis that would den^tistrate success. A minimum require- 
ment for this evidence was that soeie baseline* or 'comparison be- provided 
80 that gains made by the prograostudents could be clearly attributed to the. 
impact of the bilingual program. If program staff indicated that achievement 
test data .and suitable baseline data'were available for AIR review, partici- 
pat ion in the study v&B invited. 

Initial telephone contacts with non-Title VJI programs also included 
discussion of the program's soals ^d procedures. Programs were not considered- 
if it was determined that the program focus was outside the scope of this study, 
e.g.., the target group functioned well Enough in English that the native 
language was not required for instruction in pther subject areas. 

In addition to' soliciting the ^Jarticipation of programs in the study, a 
major emphasis in the telephone conversations was to establish a feeling of 
good will between program staff and AIR. If programs appeared- to be eligible, 
they were enthusiastically' encouraged to participate in the study, but vere ^ 
not preSfiured if sttfong resistance or doubts were evident. The very' few 
instances in which such reluctance was encountered were due to previous unhappy 
experiences with research studies in which. information obtained from local 
officials had been used in ways which were' detrimental to the program. AIR 
staff -assured local program officials that In the present study t\ie only pro- 
grams to be documented and publicized would be those approved as exemplary as 
a result of, AIR screening.,. «i enaorsement, and final scrutiny by the, Dissemin- 
ation Review Panel (DRP) in Washington, D.C. AIR staff pointed out that since 
this publicity would not be adverse but on the contrary complimentary, it would 
be advantageous to participate in the study. * 

Tact. was required when the information obtained duriU^ the telephone con- ^ 
versations clearly indicated that, the program should not be considered further,, 
either because of lack of dat^, inappropriate focus, or serving a target grOup . 
that, though bilingual, did not require bilingual ins'truction. In these cases, 
the reasons^ for not pursuing the program's candidacy were explained and agree- 
ment was reached with local staff to drop the program from the study. In 

1 Although care was taken^^lo^'explain requirements of the study and to solicit 
documents only from appropriate programs, efforts were also made to give the 
benefit of the doubt and not to diacourage any program' that might be a poten- • 
tial candidate. *Aa a result some of those that did submit documents also 
had to be rejected for the above reasons. 21 ' ' 
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every case wher4 this was necessary, no hard feelfngs resulted and local staff 
- . were -in complete agreement with the decision. A total of 59 programs were 

droppe-d from the initial candidate pcrol on the basis of preliminary telephone 
• ' contacts or letters requesting nor to be considered for the study. 

- . Thirty-one programs included in the study were 'not initially contacted 
. by telephone. These 31 were Title VII. programs for which recent,. evaluation ^ 
reports were reviewed by the AIR project director at the Office of Education in 
Washington! D. C. soon after contract award. 

Acquiring Program Doq'uments , * 
If answers to telephone queries were affirmative, program staff were ■aske4 
to f onward documents. In several instances, local staff requested AIR to send 
a confirming letter which described the study, reiterated data requirements, 
and requested*the desired documents. A copy of this letter of confirmation is 

included in Appendix F. 

A follow-up letter was .mailed to programs. that failed to 6end the promised 
documents within 10 days. To encourage programs to send documents even when 
'local files were reduced to single copies, the letter indicated that^upou 
request AIR would xerox and return the reports. -A copy of the follow-up letteif 
is contained in Appendix B. A total of 47 .follow-up^ letters were mailed/ 
Documents were sent by 65' programs; 20 did not send them. Adding the 
31 programs for which documents were obtained directly from the Office of . 
Education, the total number of progr^ for which documents were reviewqj vas 
96. Table 1 summarizes re^uj^ts o£ identifying candidate programs. 

1, - Table 1 
Summary .of the Results of Identifying Candidate Programs 

Initial Candidate Pool >^ 

Programs identified by screening documents at. the 

Office of Education, Washington D.C. ' ^ * ' 

Programs identified on lists, supplied by the Office of . t*A' * 

Education or by leads supplied from other sources , . I51 

Total Initial Candidate Pool ' 175 

Drops from Candidate Pool 

Dropped during initial telephone "contact or in 
. response to program^s written request 

Dropped because documents were not received ' ^ 20 
Q , • '25^ loiol Drops 79 

ERIC Renalnlng Candidate PrograBM * . * 2^. 




Doeuaent Recalot and ConProl Pr ocedures 
- The main co«pon«tits of tht system for maintaining information* files on 
candidate programs w«?ra theae: ' •' • 

. • Call/Contact Pile 
Card File 

Document File . • '* 

• Alphabetical Notebook , ^ ' . ' 

Call/Contact File ♦ . 

A folder for each candidate program wa<^ maintained in the Call/Conl .ict 
File. BBch folder contained a rccor.l oi -aH <..ntacts with the proKram prior 
to the receipt of documents, and a sheet indicating documents- received. Vox ■ 
example, initial telephcTne conversations with each .program' s director were 
summarized in writipg on a special form wtUch was filed in the folder. In 
cases of programs that were dropped on the basis of these c;onv-er.snCions the 
contact form indicates the reason for this actictfi and the name and location of 

■ the program contact.: If the program asked for a confirming letter before 

■ supplying documents, a copy of the correspondence was ^iled in the program 
folder. If a follow-up letter reiterating the request for documents was re- 
quired, a cop.y of this letter was filed in the program folder. When docyments 

. were received, a(^heet identifying thenf was also placed in the folder. _ 

Card File ' ' . . ' . . . 

The main pGrposes of this file were to identify the number and years of ^ 
evaluation. reports provided by a program and tp provide an efficient check-in _ 
and check-oli.fc' sjrstem. . ,' , • 

' ' When a document was received, a- card wag typed showing the program title • 
Lid location and the name"^d, address of the program directof or other' contact. 
An identification number was' assigned to the program and noted in the upper.,, ^ 
right-hand comer of the card. Also noted were the academic ^^^^^J^^ "^ich ' , 
eva:iuatio.n reports were sent, and if the program requested that trt? documents 
be returned. . * " ' 

' yrhenever documents were given. to an AIR staff reviewer, his or her initials 
were noted- on the document card and were crossed out when the documents were 
returned. The system was necessary because documents were passed around during 
the multiple review process, and it was important to be able to tell where they 
could>e found fet any point in timeJ For example, when , a program's status was 
being queried, the card was examined to see which reviewer currently had the 



'docun^ats .nd h6w nany other reviewers had eWtxed thea. . These individuals 
could than -be consulted regarding the program's standing in -the reviewed-. 

: ■ • ^ / ■ A . ^ • ■ 

screen cycle. 

, . card, were arranged numerically in this file, according to the program . 

identification nua^er. ^,>f . • » 

• DocuiDent File « ' ' 

The 'Document File consisted of large manila envelopes, each of which 
contained all documents for a particular program. ' On the outride of t^he . 
env^opevas noted the program identif icat ic.^nun.her. program clM- and loca- 

yeavo eovorad -^^^ev^Uon documents in , lu- 'envelope. 
The envelopes were ^iled nuinerically by program identification number. 

Alphabetical Notebook ' ' ^ 

' As each program dard was typed for the car^ File, it was xeroxed on a . 

■'sheet of paper which was .^«Hd-^---i^<^---i^-^i'^-^^^ 
'bookl..:J3ie^^hee4 each program was filed-alphabetically.' first ^by state. 

then bjr site, and finally by pro-am title. , ' ' ' 

. The purpose of this notebook was to enable AIR staff Co find the identi-^ 
fica'tion number for a program- when -only the program location .was. known. Fo,. 
example, if an AIR reviser naeded" pro grain documents for the Title VII . 
iUngual Education Program ifn Corpus Chfisti? Texas, he/she first had to know 
the p^r^entification number .becaus^e documents were filed numerically by. 
-^his number. ^S^iliT^-h^e viewer would first check the Alphabetical Notebook 
•Ider Texas, Crpus ChrlT^rr-t^ind the program identification number. Using 
r'iSs number, he/she" would i;ok . 7;r^.>^B in,the Document File, , If documents 
'"were not-there. t^e reviewer would use • tT^^.mM£-^if ic^tion r^mber to locate 
: the check-in/check-out card in the (iatd File. -Do;;im^ could then b. retrieved 
from the ataff -tiiember who had checked' them out. 



CHAPTER' IV 

APPLYING CRITERIA TO SCREEN CANDIDATE PROGRAMS 

V * , ' ' • 

Document Review and Telephone .Follov-Up 
kiiet program. documents had been received and lo^ed In, each program *• 
ev^^^tion passed through an Intensive review process. Reviewers looked 
first at the 69St recent evaluation reports. available for each program;^ in 
most cases this was *f o*r the 1973-74 school yeaf. If a program report for- 
that year*was missing and earlier reports looked promising, phone calls were 
made to obtain more recent reports. If ^ program had sent only the latent 
-report or interim reports that appeared promising, phone requests were made 
for earlier reports so thats^iewers could examine longitudinal trends. in 
program data. Throughout the entire review, as questions arose thaf could 
not be answered by reference to the documents in hand, telephone contacts . 
were made to, get specif Ic 'answers or to request more Ihformatlon by mall. 

The Multiple Review System . 

The screening- of program evaluation documents involved a multiple revle;^ , 
system. Evaluation reports for a program were given to one of four senior st.i/f 
members who reviewed them on the basis of the criteria for exemplary status* that" 
had been eS.tablished. Each reviewer evaluated the, report independently, using, 
an ln-ht)use ratin^^^oTO to record information and judgments under the follow- 
ing categories: * , ' , 

•* Comparison method used In program evaluation ^ 

• Measures' used and content areas measured 

• Sampling (attrition,' conditions of exclusion,- size) 

• * Levels (grades or iges -Included in evaluation) 
y • ^^tatistics used , in analyzing data 

#' Slgnif ic^rfce iatatistic^l and educational) 
('These oategori^'rare discussed in defcaU in a JLater section of th^s chapiter, 
and a copy of the rating sheet appears i'n Appendix C.) ^ 
. * After indicating a yes or no decision on the rating form, reviewers passed 
J:he program documents 'on- for another Independent judgment. Rating *f oris were 
top t^ by tTie reviewer who directed the evaluation activities. 

Inadequate pr<!>gra;D evaluations were relatively easy to reject; the better 
and more comprehen^e ones deceived more intensive scrutiny: When programs* 



appeared.' promising, or. when jodgnents' differed, reviewers met together to . 
compare notes aod diicuss conflicting judgments, , 

^- This procedure was deeigned to ensuri that ea^ progi^am evaluation was 
selecte^d or rejecW on .the bas.is of.* more than ohe' opinion .'• l^weyer. it soon 
became tppate'nt :that the majority of .program eval^tions fell ^hort o€ meeling 
essential, criteria, and that it>.i^nefficient .to -rl^iptre , independ^t -ratings 
trom^our senior staff mgmbers when reasons-for rfej^tion were so basic that 
sophisticaced analysis was Unnecessary. The procedure , was therefore modified 
so- that- on/ staff member was given the- f ull-pime' assignm^t'of reN^iewiT^g each 
'program evaluation in depth and producing a r^t^ing" fotm'on which wer« recorded 
'a.yes or no- recotHmendation and, for the" 'guidance ^pf subsequent reviewers, a 
"cD^pl^te listi^g.of rele^lnt information under each of the six categories, 
'together with 'references, to the page number's in the reports where that infonna- 
ti'onVould be found, thus £he first' rating form complefed for. each program . 
beciime'a resource for all reviewers, evea' though they continued to make, their 
•judgmehts independently'and to use a separate -rating fonn f or --recording their 

own notes and opinions. ^ ^ . 

■ After this preliminary review, .program evaluations were generally passed 
ort to the-stady's director of ■ evaluation, 'regardless of whether they had re- 
ceived a'yes or no : recommendation. The only exceptions were '23 program evalua 
tions-with flaws so serious that there was no point in fuifther review, In the 
majority of these cases, the director of evaluation saw the documents first 
•and ma^ this d'ecision. "shortcomings which were considered serious enough to 
'justify rejection after only one review were the following: 

• The evaluation addressed effective aspects only. 

• There were no' data x)f any kind, either because the ^program ^ 
was too new or because the documents were not evaluations 

' t but .rather descriptions or proposals. 0 

• .^Tests Were given in only one language. ^ . 

• comparisons were possible, 'either^ecause the prog'^am 

''used criterion-referenced tests or teacher ratings only, 
or used non-standardized tests with no comparisdn grpup. 

• Evaluation was based on an unacceptably small sample (less 
than 30), or on a clearly non-target sample (e.g. English- 
dominant children only). ' 

The project's director of evaluation completed his independent assess- 
ment of the documents, using the reference- notes of the first reviewer tp 



guide -hin quickly to the pages where selection procedures were described, 
measures used were listed, test data and. statistical analyses we^e presented, 
and results and conclusions were discussed. 
. , If both the first and second reviewers gave a prograo a negative rating, 

it was rejected. Jf both rated a progran positively or if their opinions 
'differed, the documents were referred pn to another reviewer for" his opinion. 

, I 

The Ratltig Sheet 

A rating sheet was developed for use both a quf ck checklist md 
opportunity for brief connnent and a prelinin<ir% decision (except vlu-ro a nvy^i 
tive decision waa clearly appropriate). The hendingp of th^> rating ^-^^^ ^'t 

/ 

discussed below. 

• Comparison group — The jy)eri'flc comparison nethod used w;v, 
n^oted. This could be a control -i^.^atmeiu comparison, a comparison 

^ with national norms on a standardized test, or a comparison with 

typical performances before implementation of the program with 
growth rates of cohorts prior ro the change in method, or a com- 
parison with any other standard—appropriate or inappropriate-- 
■ used or implied by the%valuatoivs or report writers for the pror.rnm*. 
If the report, contained no specific mention of the method used, but 
reference was made to standardized tests, to grade equivalents or 
to percentiles, reviewers would-enter "national norms." 

• Measures— Under th^s heading, use of standardised tests 
such as the Metropolitan Achievement .Tests , Stanford 'Achievement 
Tests or Cooperative Test Series was noted; also noted were npn- 
standardized tests -such as the Inter-American Reading Series (al- 
most the only choice for Spanish), or locally developed tests and 
criterion-referenced tests. The subjects for wliich these tests 
were used were recorded, such as English, Spanish, mathematics, 

- . social studies, or science. Affective measures were also noted. 

. • Sampling (attrition, conditions of exclusion, size) — The 
generalization of findings! as well as their trustworthiness, are 
dependent upon the soundness of sampling procedures. Under this 
heading, reviewers noted the methoji[s used for the selection of 
treatment groups,^ ancf*^ where appropriate, of the control" groups. 
Reasons for excluding students from the evaluation sample can be 
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■SB inportant as those for selection, and for this reason 
reviewers looked for indications of sample losses and the 
steps taken to avoid resultant biases in findings, f 

• Levels— Reviewers recorded here the grade range to which 
the treatment toas applied, e.g., kindergarten through grade 4.^ 

• Statistics— The information noted here covered hot It the 
fora of ^hp original data (e.g., r.-^w scores, mciar.i s, on-., 
percentages or proportions) and the -^tatistica; m.-th.Mis a.vi -^-nnMU- 
lations used for depcriptiv^ puri-o.^ -s - r f or , i crenr. . 

4 

• Significance— Cla ins, usu.HIv based <.,, ' Uv res-ili • < K- t 
t-tpsts, or of a 'non-paran>etrir ..t,-.t j i' ■ ' t-^' ^-r- notcdi Ti.-n, 
with the reviewer's comments or Interi.rtt.it ion<^ . Reviewers ..-ro,- 
nized that with large 'samples , trivial differences (In the sc nsr 

. of being of little or no practical importance, e.g., less th.in .1 
third of a standard deviation) -will often reach prescribed levels 
of statistical significance. 

• Decision— The reviewer marked his rating •shee.t with "yes" 
if no serious objections to the program were found or with "yes (?)" 
if he or she had reservations. "No" meant what it said^ but 

"no (?)" t^eant that the reviewer felt that he or she could per- 
haps be persuaded to vote for the program if other reviewers 
found suitably compelling reasons for approving it. 

Finally, comments were added to the rating sheet, for sample, to obtain 
-telephone clarifications of questions and missing information that were needed 
before validation review couid- proceed. 

^ Review sheets with decisions on"them were not circulated to other re- 
viewers to avoid influencing their judgments; page references and factual_^ 
information were provided to speed their review task. 

I • ■ 

Selection of Programs for Site Visiting 
The criteria established for exemplary status (discussed in Chapter IT) 
were applied to screen program evaluations and choose those worth site visit- 
ing. . In applying the criteria, the followTHg minimal requirements of success- 
ful evaluations were considered, 
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Cgaparlson Group 

Soae kind of conparison of progress of the bilingual group with that 
of a non-prbgran group had to be nade or Inpliedr Mere demonstration ^f ^ 
gAins of onspecifife^ size (e.g., significantly higher raw scpres at the 
posttest than at the pretest on a loc^ally developed measure) was insuffl- 
clent evidence of prograo effectiveness. 

Measures ^ ^ , ^ 

Although the^use,of well-known stand^dized tests'!)! English and 
^thematics, with adceptable validity and reliability, was preferred, con- 
sideration was given to locally developed measures if information on validity 
and reliability was provided and if progress on this measure could be meaning- 
fully compared wi^th a non-program group. Standardized measures of performance 
on the languages other than. English are simply not available; nevertheless, 
reviewers examined approp/iate measures when tt^ese were reported. Measure- 
ments of affective attributes (e.g., parent satisfaction or pupils' self- 
concept) were examined, but were judged. to be insufficient in the absence of 
cognitive achievement data. Criterion-referenced tests were frequently used, 
but in no case was a program evaluation found which made meaningful compari- 
sons possible. 

Sampling (Attrition, Conditions of Exclusion. Size) . i 

There were several important considerations. 

• The program group should include at least AO pupils 
enrolled in at least two classes. In the end, however^ 
reviewers were unlikely to pass any program with fewer 
than 100 pupils enrolled in fewer than five separate . 

« classes. * 

• No mention of Attrition, or an unusually <high rate (say, 
about 15%), called for. investigation. Revi-ewers "^were^ 
Inclined to reject programs with much higher .attrition 
r^tes, since biases at .this rate can exceed a third of^ 
«a standard deviation. ' ' 

• Specific exclusion of defined target groups was tenta- 

' ' , tively regarded as a negative indication,' such as con-^ 

fining the reporting of results to students "whose English 
. was already fairly good. 
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Levels ^ . 

While it was sufficient for a program to operate at. a single grade level, 
*in such a case it had to operate in several classes under several teachers. 
Samples per grade could be relatively small, "if several grades were involved. 

♦ Statistics 

Since processing raw data vas outside the scope of this study, the pro- 
grams reporting lists of students' names with scores were rejected. Data 
requirements were flexible, however, requiring only that some form of descrip- 
,tive summaries be given. -These could be in raw score units,, or in scaled 
score units. There had to' be some measure of .xentral tendency such as mean 
or median, /or a proportion; and there had to be some measure of dispersion, 
such as a standard deviatioa, or interquartile range, or some data by which 
such a measure could be sljnply derived, for example, by inverting a t-test, 

In sime statistical summaries, reviewers were able to derive oth.er needed 
statistics where these were not given. In some evaluations, analyses of vari- 
ance and/or covariance were reported, not always with justification and some- ^ 
times incorrectly. Minor errors were overlooked, .but reviewers would not allow 
■ sophisticated analysis to cloud issues of poor design or inadequate data. For 
example, adjustment of means through analysis of variance is common, and suit- 
able for equating small group pretest differences, but examples of adjustments 
were encountered when control groups differed from ^^erimental groups at 
pretest by as much as a half standard deviation. If the effect was to make 
subsequent testa more conservative, it was allowed; otherwise, the program 

♦ was treated with caution. 

Where possible,.- reviewers looked at the progress of cohorts year by year, 
l • and at all grades for a single year. Some progressive losses were expected 

♦ and found; the data were searchid to- determine whether^ these progressive losses 
were no greater than before introduction of the new progrAi* 

S ignificance c , 

Reviewers looked for the. following acceptable possibilities regarding 

Statistical significance. ' ' ^ 

• The treatment group showed' significantly better gains than 
a control group. 

• The treatment group showed significantly' better gains jthan 
. ' similar students in the same grades prior to the intro^uc- 

tion of ,the program. ^ 



ERLC 



33 

22 



.. The progr^ students showed significantly greater gains 
. "than they achieved prior to entering the program. 
. 'The program students' achievements relative to grade 
equivalents or percentiles were significantly higher 
- in present grades than they had been in earlier grades • , 
' without* the program. . ' • . 

. . The im{>rovement in achievement was large enough to merit 
attention. Reviewers looked more closely at programs 
where comparative gains were at least' a qoarter of a 

standard^eviation or bettef. although better than a 
. third of a s^^ndard' deviation was considered minimal for 
an outcome to be educationally important. 

^^^"^'^^ * . J ^ oil rhP crlteri-a foT sound evidence 

• None of the 96 programs reviewed met all^the criteria 

^ i ^Ko-i^ ovnluation <iocume;its and contacting t 
of effectiveness, based on examining their evaluation <iocuia ' ^ • 

: jrilr clar.n=acio„. « .He.e ,e. onl, pro,.a.. were finaU, Jod.ea 

to ».rlt alt. vUltlns.- The result, of pro.raB scraenlos are sho™ In the . 

Miater Li,; of Pro^raaa 'i. Appendix A. with an indication of why excluded pro 
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arams were drojiped. ' 

- The a^reening process described inthis chapter focuased on program 
evaluation and the quality of evidence supporting progra„ impact on parti- 
cipants. Chapter 11 also referred to criteria for reasonable program cost 
an clear and describable progr., processes. ^Hini^al re,u-ire.ents for « t- . 
i„g these criteaa could not be specif led as clearly as could re,u r»ent 
f„ sound evaluation.. Turther^ore. progrs. doc^enta seldo» included Cos . 
information, and they presented few, if any. descriptions of instructional 
and management processes. Tor these reasons. progr«. P'oc-"-" = • 
„e;e not examined closely until site visits were »..de. The next chapter, 
discusses the site visits- to the eight programs selected, and .the outcomes. 

f ■ 

Fv.lu.tion Rev).w \v Staff Pr ^^r to Site Vlaits 
Program evaluations which passed through the review system described above 
and received positive ratings (a ■■yes" or a ■■yes (»■■) from. b,th the first and 
second reviewers, and a confirmation of this fr^ a fhird reviewer, were con- 
sidered to be- candidates for site visi^tPg. «>an several likely programs .ad 
been identified, especial evaluation r.i^ew^.eting was held to finali.e^the 
decisions as to which should be visited. There were two purposes for such . 
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taeetinge:' (l).to allow *taff members to raise the sharj^fest possible criticisms 
.of each program evalu*tioo. eo thgt untenable a'rguments could be recognized and 
programs dropped before site- visiting, and <2) to point up any questions or 
needs for additional data which might enhance the arguments in favor of a pro- 
gram, so that^'site visit teams could collect the information while on site. 
Questions frequently dealt with the availabili'ty of additional baseline data, ^ 
. information about the selection, composition and treatment oY comparison 
groups, and the possibility of obt«lnins either long/tudin. . <i.... or the mosi . 

recent data (In cases where a program had n : prei' ions 1 v f u. n ...... •!. M.- . A 

tota] of 19 programs had strong enough -..Ju.ks to .:.,rrnn. . ons:-!. .-t ion .a Ihr 
review meetings* 

• The review meetings were nLte.ided by LUe four sv-.io, ...-.I- -vk-u., .. 1 1., 
.preliminary reviewer, and the projoct dlrocor." IJeallv, I ! , .-.m . . -^wo,, id 
have analyzed the'program documents, cnpieted -rating forjn-s. .o,.l u..t..d tluMr 
questions before these meetings took place.- In actual pr.-.-ti.c. li.ero was ' 
not always time for this; however., in all cases at least C.r^e. senior reviewers 
had seen the program documents prior to the meetings. The other reviewer^ and 
the project director acted as a critical audience for the present ..tion of the 
cape regarding each program, including both its Strong and weak .point.H. 

AS a result of these meetings, seven programs were dropped from considera- 
tion because it was decided that the available data did not justify site visit- 
ing. and .that no better data would be available. An additional four programs 
that had been tentatively identified for a possible site visit were also ; 
-dropped after further debate. In the case of the remaining eight programs, 
it was decided that the evaluation data did justify s^te visiting and that 
it appeared likely that additional' supporting drfta could be obtained ^.n site. 

I' 

Pre-Site Visit Summaries 
Oncfe a program had been tentatively selected for site visiting, n brief- 
one-page summary'of program information was forwarded to the OE Project 
. Officer, so that he would be aware of basic f eatures of the progr%and the 
evidence of its effectiveness. In some cases his^review raised' additional 
• questions to be pursued on site. I'n others, he suggested notification of 
interested state education officials in the event of a-visit. 
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The pre-site^visit sumary included the name and location of the program, 
the year it began, the nam* of the evaluator and year of the report containing 
evidence of success. ' In addition, the following program evaluation features 
were briefly summarized: comparison method, measures used, sampling, grade 
levels, aspects addressed by the evaluation, and statistics used. AIR evalua- 
tors* comments were added. Finally, the tentative site-visit dates and schools 
to bt' vl-. I'led were inculcated. - ^ 

Pn — site visit summaries were suhnittc ^ for 'novo i^rr^^i > . than were 
flnnllv vi<Mt(>d. Visits to four inoprr'- v . e (-au'eH^J hin- m i- .<;.'JUonal 
inforiiMt I'-Mi .'l.iajned prior lo f i nn H / i nj^, i he dh\ n<»l sMjM>oM . irller 

»M>n< lusioiis rov^'^rdlng their ef f 0( t i v^^n^-:-, . '.i-jo, tw«» ujm ,it.i-. ^v- i • 
sclt'clod Into in the screening pro<,<-ss, .md there was i n:;uf I ic i ent time to 
prepare and subrait^^silhimar ies before xhe visits. 
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CHAPTER V 
- SITE VISITS ' 

- . * . • 

Purposes 

Site visiting provided the informa^zion needed to meet the second major 
goal o*f the study—the description of bilingual education programs which 
were recommended .to QE as exemplary programs- There were several reasons 
for visiting program sites, first, ^h'^e descriptive and evaluative data 
included in program documents had to be validated. Second, considerably 
more detailed information about the management and operation of the program 
was needed in order to write program summaries for submission to the Dissem- 
ination Revi&w Panel (DRP) and to write detailed descriptions of programs 
that were approved as exemplary by the DRP for dissemination. Finally, , 
visiting the locale, talking with program staff, and observing classrbom 
activities and management techniques provided the perspective necessary to 
write clear descriptions of program processes. 



Focus 



AIR site visitors were familiar, with the philosophy uf and rationale 
for bilingual education. An extensive literature review, undertaken for, a 
related study under the present contract, ^ had identified variables and issnes 
in bilingual education to be invesagatpl in that study. An indirect outcome 
was to acquaint AIR site visitors^^^fch aspects of bilingual education possibly 
related to program effectiveness. Discussions of certain- key program compon- 
ents appearing repeatedly in the literature didi not„specify ways in which thp^ 
might be integrated into & bilingual educ'atfonJJtogram. Site visiting provided 
AIR, staff with 'insights irtto how the rtjmpqnents. interact with each other and 
with program context as an effective educational ^process- 

To assist site visitors in collecting and organizing data, a detailed 
outline covering key program components was modeled on outlines used in ^ 
earlier AIR studies of exemplary educational programs. On the basis'l)f the 
literature review and suggestions from AIR staff who' were knowledgeable about 
bilingual education', the outline was refined to focus on features considered 
to be particularly important to bilingual education programs. 



1* This related ef fort was one of three studies' that comprised the **Evaluat^on 
oi; ESEA JitleViL B4-lingual Education Programs" funded under OE, Contract '^No. 



OEC-0-74-9331. 
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The outline was used to check the inf.onnation obtained from iiocumeats 
and" to guarantee that adequate Information was col'leict«d..whi,le. on site.. Be- 
cause programs vary so widely, however , site visitors were encouraged td allow 
unique features to emerge during interviews and classroom observations, -and to 
prohe^in depth those program characteristics that they believed to; be particularly 
significant. .Topics covered in the outline were the following:-^ 

Objectives *' * . 

Participants - ^ . ' ^ ' 

Locale 

Development * • , * 

Management • ^ • ^ ' - 

Staff ' 
Preservlce/Tn^crvKe *iniiig - 
TnstriK I loiin i A('l i H • nr.d 

Curr Icul urn - ' >' . • 

Equipment, Materials. ofkI facilities 
Parental/Community Invoivrment 

Costs ' V 

Evaluation ' , . . 

Later, to guide writers in preparing program descriptions/ this outline ' 
was modified further. It is included in Appendix E and described in Chapter ^ 
VII. 

^ . Procedures 

Pre-Site-Visit Preparation ^ ^ 

After the Project Officer approved AIR's redoramendations of* programs to 
ba visited, site vtsit teams were assigned, ani preliminary arrangements f or ' 
the visit were Lade w^th project directors, who had 'been informed of the 
possibility of a visit by AIR staff. ^ District personnel, usually 'including, j 
the superintendent, were informed of plans for the visit. When requested, ^ 
b^ickground information about OPBE's evaluation of bilingual educatiOJl and' ^ 
air's- role in this effort was ^"al so/ sent to superintendents. * ^ * ^ 
Site visitors learned as mu^ about the program as possible from docu- 
"'ments that had been.sent Referring to the outline, they noted areas 



'There were, two >xceptiops. In one case, a visit was tentatively schedul^ 
and later cancelled because more recent Evaluation reports did not substan- 
tiate earlier evidence of the programi^ effectiveness. In the other instance, 
further scrutiny of the data ^revealed inadequate evidence of the program's 
effectiveness; also, recent events had disrupted program functioning so 
severely that a site visit would not h^ve been justified. 
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for which little or no docuaented -Information was available. ^ In (*ases 
,,when there was Insufficient time before the visit for a thorough preparation, 
or- when limited- docuoentatioiv was sent by the prograa, questions' were formulated 
, . . from the outline and were sent ahiad to the director-. ^This proved to be espe- 
cially helpful in preparing program staff for AIR's visit and for scheduling 
interviews with project or district personnel who were knowledgeable about cer-' 
tain ar«as of program operations. 

Team Responsibilities' and Activities f or Site Visits 

• In genferal, each teali; consisted of two AIR staff members— a writer and 
a validator-who'visited a site for, three days. Because it was not possible 
to send a bilingual person on^each visit, AIR staff requested that they be 
accompanied on site by a bilingual member of the program staff. This individual- 
translated when necessary and explained activities being observed/ . 

^The writer "collected information necessary to .prepare a comprehensive 
program description. This entailed 'Dbservlng classes in action, reviewing 
- supplementary -documents furnished by staff', and interviewing teachers, admin- 
istrators, and parents. Writers recorded the information in two ways—taking 
ndtes and tape recording. They reviewed their notes and tapes each evening 
• and reorganized their questions based on what they h*d learfted that day. 

The validator's major responsibility was to confirm the evidence of effec- 
tiveness presented in the local' program evaltiktion reports and to 'analyze addi- 
■tl^onal d^a that had not been available prior to the visit. The validator used 
this information about the program's effectiveness to Jrepar*e'hls portion of 
the DRP program summary. . Validators also assisted writers with staff inter- ■ ■ 
views and classroom observations. , 

Schedule . ' 

. "' ^ Visits were maJje to eight programs. The schedule below shows the site 

visit dates, location, and title for each. 

, Dates ■ A Location and Program Title ^ . 

. 1, February 3-5 * La Puente, California— Project B^O 

2. February 12-lA Houston, Texas— Bilingual Education Program 

■ 3 April 14-18 Alicfe, Texas— Alice Independent .School 

District Bilingual lE'ducatlon Program 

- 4. April 14-18 Kingsvllle, Texas--Klngsvllle Bilingual 

Education Program 

! 
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• Dates Location and Program Titl;g 

** «^ » 

5, April 16-18 ' Madawaska, Maine-^-St, John Valley Bilingual 

Education Program 

6'. .April 21-23 Santa Fe, New Mexico--^ESEA Title VII 

Bilingual Education Program 

. 7. April 21-25 Philadelphia, Pennsylvania— Let *s Be Aipigos 

8, May 5-7 / Corpus Christi, Texas—Aprendemos en Dos Idiomas 

Title VII Bilingual Project 

During the eight site visits, additional inf orraat ion^ and documentation 
were obtained for each' prpgrairi. This new information w.^ reviewed to deier- 
mine if it substantiated earlier conclusions regarding progfam effectiveness. 
In all but one case, it did. 

Summaries of the remaining seven pirograms were written and submitted to 
the DRP. The nature of these summaries and the results of the DRP review are 
discussed in the next chapter. ^ 



< ' * ■ 



3 One* program was dropped after the site v^it because the 1973-74 
evaluation report, made,^ available during the vieit, did not provide 
evidence needed to support claims, that the program substantially im- 
proved the Enflish' reading and language skills df its participants- 
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PROGRAM RIw'lEM BY THE DISSEMINATION 'rEVIEW PANEL. (DRP) .-^^ 

S" • . _ : Background . Purpose, and Structure of the PR? 

'* The purpose of the DW was described in ar statenefit prepared by. John Evans 
Cha'inaan of.' the Panel, dated 15 -Novefaber 197A: ^ * 

During 1972,^ the Assistant Secretary of Kducnt H»n and 
"several 0E> s'ta.f f members 'became consri^^us that the dls- 
•semlnation actiiUies'of the vai ious OF progr.ims were 
proceeding in, a iar^r ly na' i -Mviont , an-l un- 

coordinated manner . . ,.with»th(» J nr inip]i.-d 

• reccTmmendation that these mater i.U^^ "o>:einpJ;iry , 
"promising," or^ *'ef fective." and that school systems 
would be well advised to institute' these methoas and 

• |>jograins in their'- owri schix^ls. 

A ^review, ot these disseminations; however, revealed ^ • * ^- 

' that, in' most caaes tHlerp. was little pr no evidence to 

back up Lhe claims that'<the materials were exemplary- 
: . or indeed effective .at all; and that putting^an official . 

OE* stamp of -^^^^rWal on .them and urging schools to 
adopt them was ptjemature a^^besr and Irresponsible at 

' ,^ worst. , • ■ -^^ ^ ' , " ^' ^ ' 

•Afccordingly, a Di8's^inatl<i>n' Review Vahe^l was created 
po review all' programs; 'p?t^ojects*, models, methods, . 
^..materials, eXc, propo&ed*?or dissemination to .deter- 
mine that there vas^ adequate' e<ridence of their effec- 
tiveness JJiefore-Miss'imination was gjpproved. 

^ '* .thV Panel is now a joint'iody of the Education Divfsio^ of the Department 
6;f.He"«ith, Education, arid ^Welfare, ,wi5[h its members drawn from the Office of 
Education; (OE) and tlye' Natf^nal Ins^mufee of Education (NIE) . Each agency^ has' 
H repreaentattves on the, 22-merater Panel. Staff services are, provided by, the 
"off if e Ik the Assistatit Secretary for Education. A quorum of seven members Is 
"Required, for a 'DRE'^i^fetingT^nd no ^wer tha-n three members each from OE and ^ 
"*SIB muft be, present.- ^ " ' ^ . 

' Dife membeirs' are s'fei'ected tfor their- ability vto analyze and understand evi- 
denc^of effectiveness supplied by program evaluation jiata, and also for their 
general knowledge and' experieiwte in educatioh (Evans, op. cit.). 
- ihrogram submiflsions are prepared in.a'special format according to an out- 
lixi^ lJupp'iied by .the DRP.^ A copx of the outline is contained in Appendix D. 
Isefe ."Form Ipr Submitting. Materials the Dissemination Review^ Panel.") 



Preparation of. Summary Descriptions for PKP Ravlev 
xy of , program methods and evaluation r-esults was prepared. for each 
of the seven -progrtM r^coociended as exeaf>lary by AIR and the Project^ Officer • 
The seyei;i sunxaariea arc included in Appendix D» 

Each summary has two main secTtlons- ' In the first, key program strategies 
are describe<J. These are features' that appear to be contributing;' to the out- 
standing achievement gains of participants as judged from on-site observations 
examinatlonwof- program documents, and interviews with program staff. While 
DRP judgment is essentially based on the ev^denpe of effectiveness presented 
in the Second* half of tXie summary, the detailed description of key program 
features must be compelling -and detailed enough" to lend additional credibility 
to the data provided and to assui^e replicabllity . 

The second section of the DRP summary presents evaluation evidence snppor 
ing the claitn of 4)rogram effectiveness. The details provided include informa- 
tion on the characteristics ^f program. an<3 comparison groups, identification 
of instruments used to measure achievei^ent a^d/or'^ttitiides, and results and^ 
interpretation of data analyses^ This infotmatioij is summarized in a manner;'' 
that highlights th'e rationale for recommending the program to the DRP. 

'Draft versions of the DRP summaries were forwarSed to <he Proj^t Officer 
for review' well in advance o^ the. DRP meeting^. The Project .Of fleer reviewed 
the summaries, recQmmended^^^§\tt'slons that would clarify the information pre- 
sented, and requ'ested supplementary, information on points that might be ques- 
tloned during the DRP meeting.^:' - , ' ' 

Final versions of the program summaries wece circulated by the Project 
/officer to DRP member^ and Title VII staff in advance of the scheduled reyiew 
meetings. These gummar^es-vere stu^ied^in preparation for discussing and 
judging each program's vofthiness of Wide-scale dissemination and official 
endorsement bv the Education Division of the Department* of Health, Education, 

^nd Welfare. * . . . 

Procedures and Results of the DRP Meetings 
The DRP met twice to review AIR's submissions^ four program summaries* 
weye considered at the first session and three at the second. Ttie presents- 
tionk w^re made by -the Project Officer who was accompanied by a Title VII 
T^ipresethtative, ' ^ ' / - 

./ the r^'lew procedure involved discussing the informatiop presented in, 
the program summ.ary* Questions fr^ DJRP members about specific points \werc 



directed .to the Project Officer and TitlT VII. representative. Following their 
discussion, the Panel- meiabers voted as to whether the program should be approved- 

for disaeaination. ■ " ^ ' * 

At the first meeting, three of the four submissions were approved. At 
the second meeting. "one/of three {^rograis was approved. The programs approved 
for disseaination b*y the DRP are shpwn in Table 2. 

• .. Table 2 

Bilingual Educaticfti Programs Recommended by AIR 
^ and^Xpproved for Dissemination by the DRP 



Program Title ' ' ^ Location 

St. John Valley Bilingual ^ Madawaska, Maine 

Educatibn. Program - ^ ^ ' 

Alice Independent School- Alice, Texas 

District Bilingual Education* 
Frogram ,1 . 

^\|)renaemos. en Dos Idipmas, ^ Corpus Christi, Texas 

Title VII Bilingual Project * , 

" . BlLingual Education Progr-a^i Hou ston>> Texas 

On a- percentage basis, '577. ^ AIJl's submissions were approved. This is 
comparable to the average approval rate of between 50% and 60%- for all programs 
reviewed. by the Panel since it was formed. Although a numbef of bilingual pro- 
grams have' been submitted to the Panel during this time, these four approvals 
are the first ever awarded to '^ji lingual education programs. 



CHAPTER VII • • 

DEVaOPHDiT OF DETAILED PROGRAM DESCRIPTIONS 

Purpose 

^\ A major product of this study is the set'of four detailed program descrip- 
tibns that were prepared for programs approved by the Dissemination Review 
Panel (DRP) . These descriptions vere" to prov7<lr models for project planners^* 
parent adyisory groups, tearht-r<?, ndmini';tr.-'t..r« . nml othors nnxlous to improve 
th<- odmntl«in of youngster'^ who fav Vlr.,!-.) t ' ■ •. sper-U i hr ..hility. Tlie 
program .. <?.-rYptlons wem uvi" I:.. ' ■ ! ^'^ p.i-'i'' ions ro 

b. mad. -m CO -The doc^lr.-.l-i litv ■ ! r.o.l i r I .pi ir-,t ions of 

the j.n- -.im^. and indict.- R..nr. . • ■ . ; V " .n, .Vi<.. progmm. 

For potential replfrnto, . i/*;'' ■ -1 -t approv ■ ' r "is th .L th^-Ho p...- 
^rnms provided evidence of elK t\'u;u^<. . <..„t;,lPod In their own ..v,.Juati^n 

reportfi which was judged to ho uiiicieni to jus. Uy their dissemination by tlie 

■ ■ ^ ■ . A ^ ' 

Office of Education as exempiary^bilingunJ education programs. 

\ 

• • - Content aXd Fjnphas ts 

' The Request for Proposals indicated diat tlie detailed program descriptions 
were to follow an outline expanded from a sample Included in the RFP appendix. 
•i,This outline was initially refined to provide"^^^ guide for document review and 
'Site visits, as discussed in Chapter V. Subseqaently , it was modified on the -• 
basis of site visit experience to guide writers to Include similar kinds of 
information in each detailed description. 

The outline consists of eight major' headings with several subtopics under 
each heading. Writers tried to follow the outline, and the\^ff erencfes that 
do occur .among descriptions were due primarily to the adequacy of the' informa- 
tion available and' the unique character of each prqgram. 

Below are summarized the content and emphasis of edch of the eight major 
sectiorts of the detailed program descriptions.' 

A As discussed elsewhere in this report, programs were not even initially con- 
sidered for" submission to the DRP unless their practices were consistent with 
Title VII* legislation and definitions, and unlesa evidence of effectiveness 
met criteria discussed in Chapters II and IV. The label, "exemplary,"' should 
be interpreted in the light of these criteria. 
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/ 

. Program Ovetvlev 

•In thi« iection the reader is intVoduced to the program, the iopetus for 
Its b«gitmlng, ptogr^ objectives, tcai general procedures used to achieve the 
objectives. Also included are staff and student characteristics. 

Program Development ^ ' o 

This chapter describes in more detail the reasons why the program began, 
its goals, the program's relationship to the locaJ school syst^, and aspects 
of the community that influenced planning of the. program. -If appropriate, it 
also Indicates people who wete instrumental during the planning phase and how 
' the program changed since it began. ... . 

Staffing and Management . ^ ^ 

Included .in this sectix)n is information about program staff (their quali- 
fications, responsibilities, length of tifne with the program, recruitment pro- 
cedures), staff development activities, and program management policies. 

Instruction 

^This chapter describes in detail how bilingual education is implemented 
in the classroom and includes teaching techniques, grouping patterns,; assess- *' 
ment of student achievement, facilities, and materials and equipment used. 

Parent Inv61vement and Community Awareness - , - ♦ 

In this chapter, the emphasis placed on parent involvement and/or comxminity 
awareness and examples of how parents are involved with th^ program are dis- 

CUSSQd. ^ » . , 

\ 

Costs 

An^analysis is given of bucfget information provided by the program, with 
' ' special attention paid to start-up expenses and continuation costs. Also 

given are sources and level of funding. . The per-pupil cost for the bilingual 
program is compared to the per-pupil cost for the regular school progranf. 
When possible, budget options are suggested. ; 

Evaluation 

This chapter sunmarizes the evidence of program effectiveness, indicating 
its impact on student achievement. 

Sources for Further Information 

In this final section, names are given of program staff who can answer more 
specific questions about the program; further information may be Included on the 
^ availability of materials and any references cited in the text are listed. 
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Review end Revision 
After descriptions were drafted and edited by AIR staff, they were mailed 
to the site for review by progran staff. Revisions and changes suggested by 
them were then Incorporated into the descriptions which were submitted to the 
Oflfice of Education. 
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CHAPTER VIII*^'' 
DISCUSSION OF RESULTS AND RECOMMENDATIONS 

Programs identified as exemplary in this study were those with demon- 
strated significant program ^ffect based upon evaluation designs which were, 
.sound enough to pass review first by AIR and then by the Dissemination Review 
Panel (DRP). Criteria for screening the program information have been dis- 
'^ctiSRcd in earlier chapters. It is important to emphasize that this review 
was concerned almosb totally with the quality of the local summative program 
evaluations. Per pupil costs and the descriptive clarity of the instructional 
and management procedures of a program were also examined, but aft^ confi- 
dence In the statistical and educational significance of pupil outcomes wias 
eBtablished. 

thj^s focus on the local program evaluation cannot be viewed as^ a program 
evaluation; most program components were not evaluated. In this study, the 
term exemplary may be strictly applied only to those programs showing ^signi- 
Meant effect supported by a sound evaluation and approved for dissemination 
by the DRP. For the remainder of the programs not identified as exemplary, 
*,lalms of program effectiveness, when these were reported, were not accepted 
because of inadequate program evaluations. In most cases, one simply could 
not Kionf idently conclude whether a program was effective or not. As noted 
earliei:, of the 175 programs considered for review as exemplary candidates, 
S9 reported evaluations so inadequate that no Conclusions couW be drawn about 
program effectiveness, 20 supplied no requested evaluation reports, and 59 
were dropped after contact because they were too new to haye conducted summa- 
tive evaluations or they did not meet the definition of a bilingual program. 

Since few program evaluations met acceptable standards, it is .useful to 
describe some of the more common program evaluation deficiencies in the ji)opc 
thcit, program developers might ba made aware of areas in which their ^program 
evaluations can be strengthened. It is probable that mar\y more programs could 
be identified as exemplary if the results for program participants were pro- 
perly contrasted with a comparison group, with a ^relevant norm, or with pupil 
outcomes ^rior to the introduction of the program. 
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Major Deficiencies in Local Program Evaluations 

U se of Criterion-Referenced Tests 

Many evaluations reported pupil outcomes in which criterion-referenced 
testing instruments were developed. In no case was offe found that demonstrat-ed 
prc'j'r.im effect, 

tin one somewhat extreme but not atypical example, the program developers 
^,id established unreported objectives and constructed evaluative tools ''to 
vissess .ir\ omplishment for eajch-of th« designated subject areas" (no examples 
or <J<'-.cripi ions were pn^vldexl) . Although state-mandated standardized-^'tests 
'^t-TK .1 Imliiistered to student*;, they repc^rted no data on these tests. An objec- 
iivi' s,ii(l t*> ho mastered if 70Z (^r more of the students completed It. The 
.arK» iDsi r^mviits were used for pretest and for post test after which differ- 
ent In* n/opurtions master ingv the objectives were tes^ted with a correlateci- 
?ncaju t-Lcst. Die six pupils fn grade 1 improved theirlmean .proportion o£ correct 
responst'.s frum'O.OA on the pretest for, English social studies to O.K)9 on the 
posttest wLth n t value of 3. 21, , significant at better than the 1% level. Thus, 
sU;nLf l( .ince coiild^ be claimed although the posttest proportion was far below 
the mastery level. , , ^ ' 

MrnrifMn^i spenfic obit^tives,' establishing the proportion of pupils 
whu arc expe^-ted to demonstrate learning of each of the objectives, and check- 
ing pupil objectives, and teaching methods whenever testing shows failure, 

are profitable uses of criterion-referenced testing. "Used appropriately inV 

. ' ( 
formative evaluation, this approach can alelrt program developers to one or 

more points in a program where attention is needed. 

In summative evaluation, criterion-referenced testing is useful when: 

• common objectives are set for all schools; 

• the scoring system is objective'; 

• the standards set are tied to agreed-upon typical "performances 
and well-defined conditions instead of being arbitrary. 

T ests of Significance of Gains ' . ' ' • ' 

The most widely-used experimental design is one that presumes to demon- 
strate program effect through the statistical testing of gain- scores (i.e. 
posttest minus pretest score). The statistical test applied Is usually the ^ 

4 

.cor/elated means t-test, though sometimes a non-parametric test is used. 

This prpcedure cannot demonstrate the e'ffects* of the program. At best, de- 
' / ' ' ' ♦ . 

pending upon the grade *level, a grade-equivalent gain of three to five^ months 
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over a year for a sample of 17 pupils is likely to be significant, at the 1% . 
level. This significant. g»in can be ascribed toTmaturation, or to practice 
effect; or to both, and can b« ind«pendent of the effect of the educational 

1 * — ' > 

•progtam. . , > 

One report contained no less than 204 correlated mean t-tests, mostly 
of raw scores." Predictably, more than half of these were significant at the 
1% level; only 41 were' "not significant" in spite of the fact tha^60 of the 
samples contained 14 or fewer students. Every class was tested separately' 
and again as part of the grade level. Nearly half of this evaluation report 
of over 200 pages, was devoted to this type of reporting with most .results -. 
shown in tables and bar graphs.' 

The flaw in this approach is not, of coarse, 'that there is anything wrong 
with the statistical procedure itself or even that it does not test the 
hypothesis proposed; 'the flaw is that this null hypothesis is not particularly X 
relevant-it states that no increase of learning has occurred over the period.' 
Because 6f maturation,. incidental learning, and thl development of test-taking 
"^ills, tha<<:orrect null hypothesis should be that changes in , the educational 
program have brought no c^nge in the rate of increase of learning. This 
■hypothesis is tested when, with appropriat;e care, 

• a control or comparison group is used; or 

comparison is made with the rate of increase in the same group 
before change in educational conditions; or y - » ^ 

• comparison is made with increases in classes previous to ^ 
educational change; or • 

• some reasonable basis exists for establishing an expectation 
of increase in the absence of educational change. 

Misuse of Analysis of Covariance ^ . 

Generalized versions of the t-test are the analysis of variance and the 
analysis. of covariance. When applied to pretest and pos^test scores, .the 
same limitations apply as for testing of gains. Analysis of covariance, in I 
particular, is_ occasionally found misused. This procedure_is sometimes used 
to make adjustments fot differences between treatment and comparison groups 
on pretests.' Theoretically, this makes it possible to cSjj^pare gains of dis- 
similar treatment groups. When these starting differences are themselves 
non-signlfican^ such adjustments do. little harm-, ^ut when the differences 
a(r^ large, this* adjustment is improper. « 
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' Furthemore. educational maesures are iT^terval Measures, ^p^e Is no 
direct way that the equality of these inters can be .jested, and only th^ 
most tenuous' way in which rough equivalence can be inferred through the . 
assumption of nonoar distributed measures. Adjusment through analysis of ^ 
covariaoce extrapolates the scale for the lower group .upwards, and that fot 
the superior group downwards. Thete can be no- assurance that- the two groups 
are being measured on the same scale, or even for that, matter on precisely ^ 
the same continuum' when their score distributions are located at two widely 
separated regions of the test range. . / 

This should not be seen as a crlticisia o'f the analytic pr-ocess. but' of ' 
. one use to which it is frequently" put . In general, though, the indicatidns . 
deriving from analysis of variance seem more useful as '.starting checks than 
as arguments for success. They certainly should not be used for major • 
sculpturing of unsuitable data. ' ' 

Practice EJXect 

An issue which ha^received 'rather scant attention, but which- has a , 
potentially sCr#ng iofluence on the'interpretatlon of the outcomes 
of special education programs is that of the effects of practice -on retest • 
scores in the literature, there are caveats about the increases to^e ex- 
pected from "test sophis'ticatio'n" or from "test interactions" but with no 
estlAat^s oJ the size of the effect or of its deration. .See. for example. 
Campbell and Stanley in H^ndhnok of Research on Jeacl^ng (N.I.. Gage, Ed.. 
. 1963 p. 175). Most test users are inclined to dismiss the. dangers as not_ , . 
applicable when parallel forms of thesame test are used, or as trivial when - 
several months intervene. Both sources of comfort are probably unjustif ie.d. • 
but the risks are surely greater when, as in at least one case reviewed. 
- there' is deliberate and explicit coaching on home-made parallel forms of the 
test between testings. In one relevant and suggestive experiment by Verstor 
(1974), the effects of real learning- were effectively equalized for four. " 
. samples who we-reigiven an initial test of cognitive abilities, and then given 
varying numbers .of reteSts at three-month intervals. ' The first retest for all 
•foar groups produced almost identical gains irrespective of the time interval 
between test and reLst. This gain was roughly one-third of a standard devia- 
tion;' even a lapse of a year had. very little effect... The gain for the second" 
retest was also virtually constant" for the , three "groups involved, and of the 
order of about" a qu&rter of a'st/ndard deviation., , ' ^ 
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•If pi-act lce>'f feet a'lone can cause important changes, then part of the 
anount of reported. gains attcibuCed to program effect would have to be dis- 
counted. The fffect w6uld b^ largest in the lower grades where there had 
been 1 letle -trst-tak/ijg experience; it would also 'be largest for less sophis- 
ticated students from countries with less emphasis on testing, for example, 
■ Portugnpse ImmlRr^nts from the Azores -or Spanish-speaking Puerto Ricans. 

Onc' proHr.iin tested its students twice (or pretesting, once in Spanish 
•nnd on. e-wltl; ihe KnVi.i?it. "form of the parnUel Lost; for the -p^^t test, both 
.Ungl^ish n,uj\.i...niqh -vrrslons were .iv-iln ir- i n i stered . ^Furthermore, the same 
'stu(U;u(4? ifVf'Jii'i.'ifi'ls I'l.HOss in ■ ich -uhso.j'ient y.r.'Av ^.n II. .i' l>v Krade 4 
"they VmulTl r.nvi-. l^^jVn t^^ti-d i.ioi .r \h I ime.<. flio <6mpiri>;on p.ro'up. Iipwevcr, 
w;is- dj;<wK .rao.l.nily am-w earl. . y.-nr with, in .ill probability, a Rood deal lower 
gvcr ntH" iK'tnbi^r' ■• f- tosLiny.s- • . ^ 

•'Ef fects^of' Re»Jg-l nn3 of Test Nor ms • 
, ^ — ■■ — • , ^ ^ , - 

iOver the jjears, test publishers have- sometimes -found it necessary to 
- revise thfeir norm tables; this has recently happened to -the Stanford Athieve- 
mcnc-Tests, among others. T^ere' appears to be a substantial shift in/raw score 
conversions;, the same 'raw score now reflects a "higher grade equivalei^,- more 
particularly at the upper grades where the differences can be as much as a ' 
fuil grade or more highV than on the. older norms. Whatever the reason,. the 
use of the older norms at first testing or in lower grades, followed by cori- . 
versions or new norms at retesting or in higher grades, can make the program 
appear to, be successful. WhW the tests themselves have been revisted, the 
same phenomenon undoubtedly exists, but is then even more difficult to detect- 
or to compensate for. ' " ' 

Effect of Increased Expenditure Alone ■ _ , 

Ideally, of course, benefits from bilingual programs should be attributable 
partly and specifically 'to the effects of the use'of the secoftd language, and 
increased expenditures should be warranted by this treatment variable. However, 
the additional funding has been used also for considerable improvement in the 

, facilities and materials for the program group, making it a moot point whar 
combination of program variables caused the positive effect. This, of course, 

, Is the central problem in interpretation: the' difficulty of -specifying which 
program components significantly predict educational outcomes. To ascribe 
program effectiveness to just one ci -its elements is a mistake. 
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* ' r.„.rralnt8 on Tinplementatlon and Evaluatmn . 

rnnfllcts wlr i- Mnratlonal Ideals ■ 

, ,Tha».|orltyo£av.luatlona.raviawad vara flawad bayond rapalr. Appllad 

.a,aarch is a .achnicar.pacialty,. rhare ia no good raaaon wha.avar to expac. 
.hat taaohara and educational adMnlatratora are trained and experienced in ^ 
evaluation. But it is also ^ons to aas^e that problems of evaluation would ^ 
largely, disappear with better training or .ore use of research 'P"''"-- . 
Educational practice and ai«s are often in direct opposition to the needs of 
. sound su^ativc research. For example. »hne random allocation of a sample 

to expe.mantn,-and control groups Is a powerful statistical device. -it is 
. virtually Impossible in »ost educational situations; on the contrary, place- 

^o ^^r,« nreciselv becau&e there- is a need to 
ment In the treatment group is^ done precisely ^„.,,,,tor 
eliminate a aifforence between groups. It is difficult for an 
to consider witrhholding cases for comparison, since that would produce differ- 
.edc'cf. in.'itead. of removing them. ' ■ ' 

riinfll Regulations ' . 

"Tle'rveotlon education seeks to mlnimUe differences between perf or-ances 
of groups, certainly th^ ai. of Title VII appears to be to Identify and to 
. .assist defined groups of PHPils. In at least one case reviewed, ths, purpose 
ame into conflict wUh what the court considered to be the objectives o 

TK nnr «11 failures must be laid at the door of poor design. 
ae<?ettrcpatlon. Thus, not aii raiiurea muoi. 

deseKrcgaci , , is an example of problems a program may 

The following case was n«^ique, and is an example v 
have to face from federal . >ate; and court " jurisdictions, 

Their experimental design was as" good as normal educational restraints 
permit, with use of refined statistical procedures and- with clear interpreta- ^ 
' tion of result's. The'y included a control group ^n their design; checke on 

.He initial comparability of control and bUingual groups; recorded di erences 
,n exposure to their treatment; showed the effects of the program on the 1 arn 
. ,ng of English. Spanish, and' mat1.e.atics . They stated their hypotheses before 

analy.ing-their results. While evidence for the success of this program was 
not overwhelming, it was honest and entirely credible. 

But then, over a- five-year period at least six regional consultants 
' changed the program's guidelines from a planned horizontal expansion to a 

, 1n«i-- the oattern of bussing was changed, 

.. ■ vertical expansion; control groups were lost, the patter 

reducing contadt betWeen bilingual and English-dominant students. Finally, a 
desegregation suit caused the closing of one school, a redistribution of 
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Hexlcan-Merican »t.d=„.s to predcinandv .Angle .chcoU. and ^ • . 

If-ceachlns sta». T.e progr^l staff triad to readjust .ut suffara a dra t.c 
t tn sta f. Tha, co.pansatad pUoihg -ora a.phaais on .atarlals dava op- 
: t and insar.ua training, producing V apaciaXiy trainad Miinguai taac a s, 
aii Wt aavan of the. paid fro. local funda-and lost .16 of the^ to «aa hiar 
districts whan stata Mlingual lagislation was, anactad. Tha af act 0 . haee 
contlicU^ an-d changing daciaions..uat hlunt tha .ain, thrust of tha bilingual 
program effort. 

Recommendations 

Finding only savan bilingual program avaluations out of naarly .100 poaal- 
billtias to racomand to tha Mssamination Raviav Panal -as not an ■ 
unaxpactad rasult. Tha racord is no »orsa thap.haa baan found in savara 
a,alations of progr^ avaluations carriad out in aarliar atudias. condu ad 
for the OCtica of Education by AIR and othara. Practically avary study of , 
this typa ovar tha years troi savaral raaaarch organizations and across a 
yariccy of educa'tional programs, including c.„pansatory education, "^""^ ' 
programs and now bilingual education, has pointed to poor -P""-™" 
L the lac. of planning tor evaluation. • to inappropriate use of stati tical 

thods. and to a general lac. of evidence one way or .the other. AIR' a experi- 
ence in this study of finding fewer than 5!^ of progra-a which are in receipt 

public .onias,.and have evaluation "studies jood enough ^^'l^ 
wor hwhile. is by no ».ns uni,u.. For a.^. "R reports ^ - 
ChalupsXy. and Robert. (1968); HawKridga, Caapeau. ^aWitt .an " ; 

J ^ 11 nq7n- Bowers. Campeau, and Roberts (1974), also 

Warco Campeau, and Tallmadge (1971), uofwers, v-mup 

al-Lc rap rt by' Tallage- (1974) . In these st.udies, similar standards war 
:;:iied in acLaning local progra=. ev.luationa and a .i-ilarly s^l fraction of 
candidates survived this Close scrutiny. . . 

The findings and conclusions , discussed .earlier in this chapter suggest . 
several means for improving chances that .loc^-l evaluators of good projects . 
„ilf be able to develop- evaluation .reports more likely to satisfy criteria 
of program effectiveness applied in ttt-is study. - ^/ 

■ It would be a real advance if a substantial reduction could ..be jwilfe in. 
the number of programs now being rejected for lack 6£ evidence, even if this 
.eant an increase in the number ,dis<,ualif led by contrary evidence, this would 
at least mean an increase in the number to^which serious consideration could 
^ » be given, " 




r.ntdellneB' to SrK»«l Mitrlcts - Evaluation 

1.0 innovators should be required to incorporate acceptable designs for . 
evaluation in their plans vh*n. applying Tor funds. 

2. Many program, will experience considera-ble difficulty in developing 
■-sound evaluations either because they lack-the training or because th.y ^ 
encounter constrait^ts . They need Vaa^uranoe th'at .their problems are under- , 
stood and tha't even the -most difficult.circumst.nce3. caa b^. made to yield., ^ 
some useful indicators: they may need brief help from a consultant. 
' 3 nuldelines*^ould stress that relatively simple but careful designs 
can accJmplish successfully wha|:^the most complex statistical analysis often 
cannot.] Important changes." vhen their observation Is planned, can be demon- ^ 
*4^stcatedUtWut resort to a great,deal of arithmetic. _ . 

. ■ 4 Some programs are having much difficulty In finding suitable ,^Ks in • 
i.gllsh. For example, monolingual student. In fourth or fifth grade whq^:are 
beginning to learn English could be tested with standardized tests intended 
. for the second grade, but would find the content tomewhat beneath^them. There 
■ is a need for some special tests, or for advice on substitutes. 

^ . 5 Effort. Sn test development should perhaps be conscflidated to avoid" 
the prolfferationof iocal tests without validity and reliability data .or ' 
possibilities of comparison. In any case.- guidelines or consulting services 
' should" be provided to ensure that specially developed test, meet these essen- 

tlal requirements. , . ^ 

6 Many pro.jams could profit from .some guidance on the appropriate 
uses of criterion-referenced testingr.and necessity for more unifohn criteria 
when it comes to justification of their finding. ^ ; 

• 7. Also m connection with designs' for testing, most Innovators seem to 
need information on practice effect, and on .the records^ that 'should ^be. kept If 
' they ar. to avoid spurious dalins of gains. Coaching .on ^ tests is another > 
aspect of the same problem, and theMlmiting conditions under .which this is 
a legitimate device should become more widely known. 

• Guidelines 'to Evaluator s ' on Analysis of Data . ^ v ; 

-1. Evaluators. whether . from/school districts or outside consultants, 
should -outline their- evaluations alt the beginning of the program. Their ^ 
' "heeds should be allowed to influence the collectioh of data. ^ , 
2. EvSluatdrs should have contact with Title VII. even if only through, 
correspondence. They should be apprised of the kinds of evidence, called for. 
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the wei'ght th4. 8uch,»yidenc* "would. c»rry, and the format of their presen- 
"tAtioM which ■would .simplify the. tatk df readers, of their reports, 
i,, 3. TherTihould Jje.gufdelines oa--acceptable sao^ie size^ nufaiberdf^ 
.class^^ needed,. che-"att6t^feion ^Xo b*.;given_tp a5trition..and' pther W^top losses, 
^nd^thft-^ondicions ^ndeY which compartsons ca^v be recosniied' . 
* • ^ their reports da nqt to be overly coigplicateidT Sophisticated^^ 

-analyses can be more -mi isJnce than thej? are worth if repW'ted in too much 
' detaf^ or without tra^s-latioh ^nd dilcurslve treatment", f Some t imoa they are 
eompiciclv suiscrfl^uous.. 'in particular^ -anaiLyses of variance nnd cov'nriance- 
shouJd h(-ir<>;Hcd .isvtoolV^;. not products.^ ' ' . • 

■ ;>. sipnlf irnnrP te^t'inR^ particula* ot yM'nff in K- iw. toll- 

ing.''/ should not have- undye "importipceWfkche^, to It., On the other hand, no 
final report should, omi-t y. ; . • . -. / 

• mo.T;urcs of central "tendency su^h as means, medians, .or 

proportions; ' 
■ . • measures of scatter about such points, like stafdard'deviatipns , 

. * ' quartlles, 'or ranges; . # J . / 

• some means of translating these"" dimensions into^understandable 
^terras, aike'normsV stanines, grade .equivalents , percentiles,^ 

or even units\With acquired meaning; * • . ' 

- • some benchmark by which tbe. Value of the change can be estimated, 
such as the perforinarice of a -previous class, of a comparison 
group, or of the same group und-er earlier conditions..' 
" - • 6. Statistical analyses of affective measures or* opinion surveys can 

be developed in. addition to those of achievement, but the weight they will ^ 
"^ .car^y 'will, usually be' small; evaluatofs should be aware that both the objec- 
tivity of the scoring systems and the attention paid to no^-responses will be 
8crutiYii»ed before the»levels of signihcanae.a^e claimed. 
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Appeodlx A Is a xraster ll^c of all che progress cbac vere concacced «s 
poceocl^l cAodldaces for chls study. The ch«rc Is alphabetised by^«tate, city, 
and prograa title, and Izvilcates the laagtxage of the ^arg«t group* The I&st 
colissa shovs the result of screeaicg eAch of these jxrograss, nsixtz the follcv- 
isg ahbreviat^lon codes: 

1^— The nusb^g I, after a progras ioHcates that as a result of ' 
infon&atlon obtaloed during the initial telephone contact, 
the prograa vaa dropped vlthout requesting evaluation re- 
ports. Hany prograa directors indicated that no data on 
student achieveaant could he provided hecausa cha prograa 
vas new or the cvalxiation Cycle vas not yet cocplete. In 
other cases, initial telephone contacts revealed that the 
prograa did not aeet ths definition of hilingual education 
necessary to be considered for this project, e.g., the tar- 
get groap already functioned in English and no use of native 
languj^e vas necessary, for their acadeaic progress in' other 
suhj-ect areaa. Id a fev instances, calls vere nade as a 
result of false leads and no special prograa for non- 
English-dcxsinant students vas operating* Finally, sooe of 
the prograas vere dropped at their ovn request. The msiher 
of prograas receiving a 1^ vas 59. » 

^--The nisaber 2 after a prograa indicates that evaluation re- 
ports vere requested but vere not received. The niraber of 
prograas receiving a 2_ vas 20. 

2 — The nuaber 3^ indicates that the prograa vas revicved by AIR < 
staff but that the evaluation nechodology vas so inadeqtiate 
thi^t a conclusion about the prograa's success or failure 
could not be dravn. Sooe of the oore cocoon shortcoaings 
encountered in revieving evaltiation designs verts the follow- 
ing: insufficient or inappropriate comparative data, saall 
nuabers of participants and/or control students, unanalyzed 
data, data reported for one grade level only, inappropriate 
testing procedures, and. failure to collect, in addition to 
data froQ language tests, data froa tests in other subject 
areas. In a fev instances, prograa docuaents supplied little 
or no infornation on cognitive achieveaent of participants.. 
The ntober of prograas receiving a 3^ vas 89. 

REC — This abbreviation oeans that the prograa passed the revlcw- 
acld-screen hurdles and vas recocsended to the Dlssealnatlon 
Revlev Panel for consideration as an exeaplary bilingual 
education prograa. 

AP?— Indicating approval by the DRP, this abbreviation aeans that 
the recoc=Dendatlon subaltted by AIR and the Office of Educa- * 
tlon vas conflraed after a final revlev by this group. 
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yASTSR LIST 0? PROGRAKS . 



■ E2AS0H FOE 

PRO&RAM PROCSAK • , '^UKGCAGS Or SXCUJSIOS/ 
LOCATION TItI£ TARGET GROUP SELECTIOy * 



AIASKA 

Anchorage 



Developacat bf Co=uaic«tioa 
Skills with 2=f?ha«i8 on Oral 
Lftogiuge eod Reading 



Znglith 



ARI20!iA 

Ganado Bilingual Program Kavajo 

Kogalea i !«ogal^ Sle=#entary 

Siliagiil Pr^ect Spaaith 

Phoenix Phoenix Union High School 

Diatrict Bilingual Progras Spanish 

Phoenix Roosevelc District 66 

Bilingual Edupa^ion Progras Spanish 

Rock Point Rock Point Bilingual 

Education Project Navajo 

Rough 'Rock Bilingual Education *?rogras Kavajo 

San Cs^loa Bilingual Prggraa Apache 

Tucson Bilingual Bicultural Project Spanish 



CAUrORKLA 
Auburn 

Brentvood 
Calexico ^ 

Cerritos 

Chula Vista 

Cospton 

Crescent City 
Cucaaonga 

Culver City 
El Monte ^ 
Escondido 



Upper Valley 

Intercultural Project Spanish 

Project Aaigos Spanish 

Calexico Intercultural 

Design Spanish 

Title VII Portuguese 

Bilingual Prograa Portuguese 

Project Frontier, Spani^ 
Title VII Bilingual/ 

Bicultural Project Spanish 

Bilingual Education Progran Hoopa 
Title VII Bilingual 

Bicultural Education ' Spanish 

Spanish Isaersion Progran English 
Bilingual/Bicultural Education Spanish 
Bilingual-Bicultural Education Spanish 



* See page A-1- for 
explanation of code. 
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KASm UST OP PaOCaAHS (continued) 



P&OGSAM 
LOCATIOX 



P20GHAM 
HTLE 



LAliGQAGE OP 
TAECgT (SOGP 



REASON raa 
Exaosios/ 

SELECnOS * 



CALIPORKU (continued) * . 

EurtkA Horcbftm Indian California) 

Education Projact 



\ 



Fountain Valley 

Gilroy 
Gonxalea ' 
Happy Cazp 
Uayvard 

Hoopa 

La PuenCe 
Loa Angelea 
Loa Angelea 

Loa Kietoa 

McKinlayville 

Kontebello 
Mountain Viev 
Orange , 

Pasadena 
Pico Rivera 

Poaona . 
Porterville 

Porterville 

Porterville . 



Bilingual Early 
Childhood Project 



Xarok, Boopa^ 
PottOt Yxxrok» 
otbar dialacta 

Span^h 
Spanish 

^Spanish 
Indian 



* See page A-1 for 
explanation of co4e« 



Title VII Bilingual Prograa 

Gonzalea £SL/3ilingxx^l Project 

Bilingxial Education Progras 

Hayvard Bilingual-Bicultural 

Project Spanish 

Klasatb-Trinity Joint Unified 

School District Indian Progras Boopa, Yurok 

Project 3UEK0 Spanish 

Bilingual Schools Progras ' Spanish 

Castelar Bilingual ' 

Education Progras Cantoneat 
Lof Hietoa Bilingual 

Bicultoral Project Spaniah 
Asarican Ixsdian Early 

Childhood Education-SB 1258 Engliah 

Bilingtul Education Progras Spanish ^ 

Bilingiial Education- Pspgras Spanish 
Orange Bilingual/ 

Bicultural Project Spanish 

Bilingual Multicultural Progras Spanish 

Bilingual Bicultur^ Education 
Para Los Estudiantes 

de El Rancho Spanish 

Bilingual/Bicultural, Project Spanish 

Cooperative Effort in Successful 
Learning Experience — State 1238 English 

Cooperative Effort in Successful 
Learning* Experience — Stat% 2284 Spanish 

Cooperative Effort in Successful 
Learning Experiance^Title VII Spanish 



ERLC 
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HASTSR UST 07 PaOCRAMS (continued) 



LOCATIOK 



Pa(JCRAM 
TITLE 



U.SGUAGS 0? ' 
TARGSX GRQg? 



HEASOH FOR 

Exaosios/ 

SSLECTIQH » 



CAtI7025U (coatinu«d) 

Slverslde Project UKIDOS. 

Kovlaivl Beights Billngu&l/^lcuIturAl 
SducAtlon Progrsa 

S«.cr«a*nto E^rly Childhood Bilingual 

Education Progrxs 



Sftn 3ernardix>o 
Coiiaty 

San Francisco 

Sap Francisco 
San Francisco 

San Ysidro 

Santa Ana 
Santa Barbara 



BiltQgual*-BicuItural 
Exchange Project (BICE?) 

Chinese B^ilingual 
Pilot Frograo 

Chinese Bilingual Project 

Project To Advance Cultural 
Opportunities (PACO) 

Media Research and 
Evaluation Center 

ESAA Bilingual Prograa 

Santa Eacrhat^ .County 
Biiin^ual Projrect 



Santa Clara Alua Rock Unified School District 



Bilingual Education Project 

Santa Clara Spanish' Dazic School 
Couity Bilingual Project 

Stockton A Demonstration Billsgtlal- 

Bicultural ^ucation Project^ 

Ukiah Biliugual/Bicultural Project 

Uniox^ City Bilingual/Bicyltural 
^ Education Progran 

' Visalia Allensvorth Eleatntary 

' * Schpol Bilingual Progr&a 



Spanish 
Spanish 
Spanish 

Spanish 

Chinesa 
Chinese 

Spanish 

Spanish 
Spanish 

Spanish 

Spanish 

Spanish 

Spanish, Can- 
tonese, Filipino 

Spanish 
S^nish 
Spanish 



^ 1 



COLORADO 
Alaobsa 

Denver 

Johnstown 



Bilingual-Bicultural Progra!3 Spanish 
Bilingual Education Progran Spanish 
Weld BOCES Bilingual Project Spanish 



* See pag^t A-1 for - 
explanation of code. 
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MASTER LIST 0? ?R0GRA2C (continued) 



> 

PROCKAM 
LOCATIOHS 



TITL£ 



BZASOVi FOR 
lAKGUACS OF EXCLUSION/ 
TARGET GROUP SgLECTiaV * 



COLORADO (continued) 

La Junta' Pro^^ct Juntos 



Spanish 



CONKSCTICUT 
Bridgeport 

Kev Haven 



Prisary Bilingual Education 
, Pro gr aa ^ Spanish 

Bilingual Pro^raa ' Spanish 



DEUWARE 

Wllnlngton 



Blllnstjal Education Progras 



Spanish 



FLORIDA 

Dade County 

PalA Beach 

Councy 



ESAA-Blllngual Project 
ESAA-Elllngual Progr'ac 



Spanish 
Spanish 



IDAHO 

Holadale 



Canyon Oahle Bilingual 
Education Project 



Spanish 



ILLINOIS 
Chicago 

Chicago 



Bilingual Education Progras- 

ESEA Title VII Spanish 

Titlt. Ill Bilingual Prograa Spanish 



3 
1 



INDIANA 
Gary 



Bilingual Early 
.Childhood Education 



Spijiish 



KA24SAS 
Wichita 



Bilingual Education Prograa 



Spanish 



LOUISIANA 
Lafayette 

New Ibex'ia 



Lafayette Parish Bilinsual 

Prograa Fre^^i 

Iberia Parish Bilingual Prograa French 



* Sec page A-1 for 
explanation of code. 
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HAST£R LIST Or P&OGBAHS (contfnued) 



PROGEAM 
LOCATIOH 



PROGRAM 
TITLE 



LANGUAGE 0? 
TARGET GROUP 



LOUISIAKA (continued) 
' New drlea^ Kew Orleans Bilingual 
Education Project 

Bilingual" Education Prograa 



Opelousas , St« 
Landry Parish 

Ville Platte 



ESAA, Title VII, Bilingual/ 
Bicultural Project * 



Spanish 
French 

French 



REASOH FOR 
EXCLUSION/ 
SELECTION 



r 



HAIKZ 

Kadavaa'jca 



Sc. John Valley 
Bilingual Prograa 



French 



APP 



MASSACHUSETTS 
Bpi^Con 

Fall River 

Fraolnghaa 

Laurence 
Nev. Bedford 



Bilingual Currlculua Project Spanish 
Fall River Public Schools 

Bilingual Program Portuguese » 

Bilingual Program of Predominantly 

Fraainghan Public Schools * Spaaish 

Lawrence Title VII Project Spanish 

New Bedford Title VII 

Bilingual Prograa * Portuguese 



MICHIGAN 
Detroit 



MONTANA 

Box Elder 

.Crow Agency**' 
Lodgegrass 

NEW JERSEY ' 
Lakewood 

New Brunswick 
^Newark 



Detro4^.*8 Innovative Conpre- 
hensivc'^rograa for Bilingual ' Spanish 
Students 



Chippewa-Cree Bilingual ? 

Education Project • Chlpgewa-Cree 

Crow Bilingual Education Project Crow 



Bilingual ^Pro^r^ 

Bilingual Education 
in a Consortiun 

Project Better Cocsnunication 

Bilingual Education Progran 



Crow/English 



Spanish 
Spanish " 
Spanish 



* See page A-1 for 
explanation of code* 
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MASTER LIST 0? PROGRAKS (continuea) 



PROGRAM 
LOCATIOK 



PROGRAM 

'title 



REASON. FOR 
LANCUAG2 OF EXCLUSION/ 
TARGET GROUP SELECTION * 



NEW JERSEY (continued) 

Union City Title VII Prograa 



Spanish 



.1 



NEW MEXICO 

Albi^uerque ' 

ArtejiA 

Clovis 



Cuba 

E&paoola 

Gallup- • 
McKinlcy 

Hatch Valley 

Laa'Cruces 

Laa Vagal 

Mora 

Pfnaaco 

Pojoaqua 

Raoah 
Santa Fe 

Socorro 
Taos 

Tucuacari 
West Las Vegas - 



Bilingual Education Project, , 
Title I BSEA ^ • Spanith 3 

Southeastern Nev Mexico 

Bilingual Prograa Spanish ^ 

Responsive Environsent Prograa 

for Spanith-Aaerlcan Children Spanish 

Clavis-Portales Bilingual ' ' 3 

Early Childhood Prograa Spanish 

Cuba Bilingual Prograa Spaniahy 

Navajo ^ 

Espanola Bilingual 

Education Program Spanish 3 

Gallup-McKinley ^ounty Schools 

Bilingxial Education Prograa Navajo^ Zuni 3 

ESAA Bilingual Prograa Spanish 1 

Laa Cruces Bilingual 

Education Project Spanish 3 

ESEA Title VII Bilingual 

Program Spaniih 1 

Hpra Bilingual Progran Spanith 2 

Bilingual Project Spanish 1 

Reaedial Reading Prograa^ 

Bilingual Educaticrn Project 

Santa Fe Public Schools 
Title VII Bilingual Project ^ ^ Spanish 

Bilingual-Bicultural Project Spanish 

Taos Bilingual-Multicultural 
Prograa * Spanish 

The Bilingual Program, 
Title I ESEA Spanish 

Armijo Bilingual Bicultural ^ j 
Prograa Spanish 



Spanish/English 1 
Navajo 2 



REC 
1 

3 

3 

2 



* See page A-1 for 
explanation of code. 
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MASTER LIST Or* PROGHAMS {continued) 



PROGRAM 
LOCATION 



PROGRAM 
. TITLE 



REASON FOR- 
LANGUAGE Or EXCLU^IONV 
TARGET GRGU? SELECTIOK * 



NEW YORK 
Brooklyn 

Buffalo 



BlllnguAl Progr&n, Title VII 

School District 13 Spanish 

Spanish ^English Dcvelopnental 

Prograa, ?itle I ESEA Spanish 



NORrn CAROLINA 
Buies Creek 



Harnett County Sursser 

Migrant Education Project Spanish 



OHIO 

Lorrain 



Lorrain City Bilingual 
Education Prograa 



Spanish 



OKLAHOMA 
Durant 

Greasy 
Stillweil 



OREGON 

Woodburn 



Choctaw Bilingual 
Education Prograa 

.Bilingual Education Progran 

Adair County Bilingual Prograxa 



Woodburn Bilingual Project 



Choctaw 

Cherokee 

. Cherokee/ 
English 



Spanish, 
Rufsian 



pen*nsylvania' 

Lancaster 

Philadelphia 
Reading 

West Chester 



Educational Opportun^tiy 

for Bilingual , Spanish 

Let's Be Aoigos Spanish' 

Title III Modified ESL 

Bilingual {^rograra ' Spanish 

West Chester's Exeoplary 

Prograa of Bilingual Education Spanish ' 



I 

REC 
3 
3 



PUERTO RICO 
Ha to Rey 



Bilingual Project 



Spanish 



* See page A-1 for 
explanation of code. 
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PROGRAM^ 
LOCATION 



KASTER LIST OF PROGRAMS (continued) 



PROGRAM 
TITLE 



REAS(2I TOR 
LANGUAGE 0? EXCLUSION/ 
TARGET GROUP SELECTION i^ 



RHODE ISLAHD 
ProYldtnct 



Providtnct'i. Plan for 
BlllngUAL Education 



Portugutaa 



SOUTH DAKOTA 
OglftU 

Pine Ridge 
Soee 

West River 



LonesAn Day School , 

Biliaguml^Progran • Lakote 

Dropout Prevention Progran English 

Bicultural Education Progran Lakota 

Lakota Progran Lakota 



TEXAS 
' Abernathy 

Abilene 

Alice 

Austin 

Beeville 
Bsovenaville 

Colorado City 
Corpue (Ihriati 

Cryetal City 

Dallas 

• Del Rio 

Eagle Pass 
Edinburgh 
El Pato 



Project HABLA Spanish 

Project ABLE Spanish 

Alice Independent School 

District Bilingual Education 

Progran Spanish 

Bilingual Education Progran 

Region XIII Education Service 

Center ^ Spanish 

Bilingual Education Program Spanish 

Brovmsville ISD Bilingual 

Progran Spanish 

CC Cabll ' Spanish 

Aprendemos en Dos Xdionaa 

Title VII Bilingual Project Spanish 

Bilingual Biculturar 

Education, Progran Spanish 

Bilingual and Multicultural 
. Education Program Spanish 

Del Rio/San Felipe 

Bilingual Progran Spanish 

Bilingual Education Project Spanish 

Region One Bilingual Project Spanish 

Dual Language Program Spanish 



3 
2 

APP 

3 
I 

3 
2 

APP 

3 

3 

. 3 
3 • 
2 
2 



* See page A-1 for 
explanation of code. 
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MASTER LIS7 0? PROGRAMS (continued) 



PROGRAM 
LOCATION 



PROGRAM 
TITL£ 



7 



I-ANGUAGE OF 
TARGET GROUP 



REASON FOR 
EXaUSION/ 
SgLECnON * 



TEXAS (continued) 
El Paso 

El Paso 



Fort Worth 
Galveston 

Harlingen 

Houston 

Kingsville 

La j^oya 
Laredo 

Lubbocic 
McAllen 

Mercedes 

MidZAod 

Mission 

Orange Grove 
Pharr 

Port Isabel 

Robs town 
San Angelo 

San Antonio 

San Antonio 
«San AnConio 



El Paso ESAA Bilingual 

Education Program Spanish 

Ysleta Independent School • 
District Bilingual 

Education Progran Spanish 

Prograda en Dos Lenguas Spanish 

Early Childhood Bilingual 

Prograa Spanish 

Bilingual Education Progran Spanish 

Bilingual Education Progra£i ^ Spanish 

Kingsville Bilingual 

Education Progran Spanish 

Haci^ Huevos Korizont^s Spanish 
Laredo ISD Bilingual 

Education Prograo , Spanish 

Bilingual- Education Progran Spanish 
McAllen Bilingual 

Education Program Spanish 

Bilingual Education Prograo Spanish 

Bill&gual-^Sducatlon Project Spanish 

Mission Independeilt School 

District Bilingual Program Spanish 

Bilingual Education Program Spanish 

Pharr-San Juin-Alano 

Bilingual Education Program Spanish 

Project We Speak Spanish 

and English ^Spanish 

Bilingual Education Program ' Spanish 

Spanish-English Experience 

School Program Spanish 

Bilingual Bicultural 

Education Program Spanish 

ESAA Bilingual Program Spanish 

Harlandale Independent School 

District Bilingu&l Program Spanish 



3 
3 

3 
I 

APP 

REC 
2 

2 
1 

3 
1 
1 

3 
3 



* See page A-1 for 
explanation of code. 
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MASTER LIST Of PROGRAMS ^continued) 



PROGRAM 
LOCATION 



PROGRAM 
TITLE 



LA}«GUACE OF 
TARGET GROUP 



REASOH FOR 

ExausioK/. 

SELECTION * 



TEXAS (continued) 
San Antonio 



SAn Diego 
San Marcos 
Weslaco 
Zapata 



Middle School Program Spanish 

Multi-Media Prograa Spanish 

Un Paso Mas Adelante Spanish 

Bilingual Education Prograa ' Spanish 

Project Language Spanish 

Bilingual Bfcultural Project, 

"Catch UP" Spanish 



2 
2 
3 
3 



UTAH 

Blanding Bilingual Education for Havajo Navajo 

Sandy ^Jordan School District's 

Bilingual^ Education Progi^an^ Spanish 

Salt Lake City Bilingual Education Prograa Spanish 



WASHINGTON 
Ephrata 

Toppenish 
Yakina 



' Title VII Bilingual Prograa 

Project BUILD - Bilingual 
Prograa 

Bilingual Education Prograa 



Spanish 

Spanish, 
Yalcioa Indian 

Spanish'* 



WISCONSIN 
Milwaukee 



Milwaukee Public Schboli 

Bilingual/Blcultural Education 

Program Spanish 



WYOMING 
A^pahoe 



B'ilingual Program 



English 



* See page A-1 for 
explanation of code. 
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APPENDIX 3 



SAMPLE LETTERS v 

1. Pros Terrei Bell 

Cosaissioner of Education* 

2. Pros AIR explaining project 

3. Proa AIR reafaaing sices to 

send reports (follow-iip) 

4. Pr'oa AIR thanking sites for 

cooperating vith project 
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ERIC 




rERRHL SELL'S LETTER 



OcPARTMCNT Or HEALTH EOUCATIO*^. MD V^'eLfARZ 



^.0 Offico of Education's Office of Piannisg, BGd^etin^ end Evaluation 
{0?32) is ccr.ductincr a:; evali^tion of the bilindtikl education pro^aa 
under riUe VII of the Elecer.tarv ana SecondaW Education Act (as 
amended). -Ihe evaluation consists of three stii:dies: an "icpact" 
ertudy for Spanish-lanciiage bili^nguai projects/which is now in a planaiag 
pha'id^ with field data collection to ta>:c plice during the ;975-76 
school year at thirty-five Title VII sites ^ tcn-noa-Title vii sites; 
an "exploratory" study fojr Kative American /pacifio and Asiin, and 
European-language bilingual prajeots, witV field data collection 'to 
ta;<e place during the vmtcr and spring c6nths of 'the current (1974- 
75) schoQl year at acproxira'tely ten TityLe VII si>tes;'and an ^execolar^^ 
study looking for e:fective"bAlinguaI approaches at ^proxirat^ly ten 
7itl<5 VII or non-Title VII sites, vxth/field visits again to taXe plactu> 
during^ the currerft school year. 

'The Office of Education's contsactor/for .this evaluation is AnericAn 
Institutes for Research (AIR), of V&lo Alto, California. AIR has 
prcpatcd a Project Su-trary tor the/evaluation, a copy of which is 
hereu'ith Qncloscd for i^ur infortytion. The AIR staff headed by ^ ' 
Mwrs .>>ralcolsi Dar.of f (Project Director) , Antonio de Porcel (Assist^t 
Project Director), Richard 2ond /(Director of the "exploratory** study) 
and John Bowers iDiredtor of thi "exemplary" study) /are presently 
.preparing lists of sites which/will -be visited and ar^as of progran 
and policy concerns for which/data will be gathered. , The Q3?BS Pro3ect 
Officer, Mr. ^ward B. Gl^ssj^n, is responsible for ^^intaining contact 
vith the .Co=dttce on Evaluation 'and Inforr^tion Sy^tcas (CEIS) of 
the Cou-Tcil of Chief State ichbol Officers about this evaluation, whiqh- 
.has beren included in the a^iual Data Accpiisition f^lan.- 



It is expectc<! th'at one 
sai^lo groups o5 pro3ect 



=ofe sites in-your Stiite wiU be ar>ong-the 
'dra-rm for the three s;tudies. We would 
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/ 

//o 



•pprtclata'your cooperation Assistance. 2 an asking htc. Glxsssaa 
OZ 0?BE to provide you with the* pertinent list of districts aad school* 
vithin your Stetft as soon as it is rcAdy. He can be Reached at . 
»ooa 4083, 400 KaryUad Avenue, S.wJ Washington, D.C 20202, or 
(ttlephoae) 202-245-7875. for -any further laforsatioa you cay seed. 

« » 

X reoognixe that t^hls is an added re^jUAst on an already heavy wor)c 
load 4a your ageacy. We do need to do this evaluation in carrTing out 
oar- respoaaihilities and hope that ^t viU not be burdeneone for yoor 
office* ■ 



Sincerely, 



Bnclpiure / 
oot CZI5 Kepre tentative 




?• B» Bell 

0. S. Comissioner^ 

of Kdncatlon 



, ?0?H LiTTEB. FO?, ??J>^SAKS WHO TOE 

A V2ITTEN 5(i.;;VtST BErOrii srSSDDSG RSPOSTS 

(Da:*) ' 

(K«^ -ftod Address; 

Dea.r ( ) : 

7hi» letter Is la respcase :c <' 

). - As part of a larger project la bUiagual education 
being conducted bv the A=:erlcar. Ir.srtitutes for' Research, we are searching for 
excsplary blilagual education programs. I would llV:e very cuch to receive 
copies of recent prcgras ^iescriptioz and evaluation reports that exist for 

^ ) . If you vrish, 

we^can copy tnls infcniatloa and return your originals. 

rollovlng the search for efrective programs, AIR will exaalne tbelr 
reports in detail aac: select ttie aos: promising sice visiting. Particu- 
lar, at tent lon Will be given to' the soi^adnest of program evaluatl^ data. A 
alnl=u= requirement for tnls eviaence^f effectiveness la that sose baseline 
or. comparison be provided so tnat gains aade by the progas students can be 
clearly attributed cb the iapac: of the bilingual prograa. 

Dr. Bell, Cossissloner of Eaucation, hat vrltten to your Chief State 
School Officer advising hlrs of tnc project. A copy of the C«=iiiiloner 
letter Is enclosed. 

Please call us collect if ,vou i)ave further questions. I appreciate very 
auch your assistance In expediting our request, Vou ^lay forward the docunents 
directly to our director of evaluation for this project: 

A. Oscar K. Roberts 

A.MERICAN INSTITITES FOR RESEAP.CH 

P.O. Box 1113 

PalojMto, California 94302 

Becai>se our pro:ect has an extretsely short timeline, we would appreciate 
It if you could send this inforaation by FIRST CLASS MAiL £n the next day or 



V 



day 

Slocerely, 



Project Director 

Enclosure . 



^3 

A«*€putl Opportunity Empioytf 



7 a 



/ 



rOLLOW-U? LSTTSa 



■ INrrnvm 



a>*tc) 



(!Une «od Address) 



De^r ( ): 

S«v«ral days *^ la a tcicphoac coavcrsstion with ( ) you 

kladly sgrccd to send us evaluation reports for < * ' )• 

We hsvc not yet received tMs laforsation, vbich is th« re«OQ for vrltlns 
to you now. 

If you foryarded this caterlal over a week ago, plea.se caII collect 
to alert us so that wc can search our files. Our telephone acBber is 
(415) 493-3550. Ask for either Kelanie Austin or Sarah Roberts, since they 
are sost fsAilisr with our file of prograa documents. ^ 

If you are sending the material Just nov, you cay forward it directly 
to our director of evaluation for this projact: ^ 

A. Oscar H. Roberts 

AKERICAK INSTITUTES FOR RESEARCH 

P.O. Box 1113 

Palo Alto, Celifomis 94302 

Thanks very =uch for your attention to this request. Our project Is on 
an extreacly short schedule, so you will understand that the sooner wc receive 
the requested information, the aore thoroughly we will be able to review your 
materials. Your' reports will reach us nuch faster if you can call thca FIRST 
CLASS, hopefully within the next day or two. 

Sincerely, 



Project Director 

t 
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7HAXK YOC t-ETTER 

wtmvrti 

PO 1113 • Psio Alto. C4»oml4 «O02 • «15)4»3^»d0 • Ctdir AtftESeAACH 

■ • ' 

(Date) 



(He=e and Address) 



Dtav ( 



Please accept our thanks for the fine cooperation you extended 
to AIR in connection vith the search for «xe=plary bilingual education 
programs conducted by AIR uaier contract to the Office of Planning, 
Budgeting, and Evaluation in the Office of Education. 

The project is nearing conclusion now and even though your progrsa 
vaa not- one of those recocscnded for cxcaplary statxis based onj evalua- 
tion data through 1975-74, you cay be interested in aore details about 
the project's purposes, nethods, and results axjd im those prograns that 
AIR rectfaended aa cxeaplary. Tbia information vill be atassartzed in 
the final report entitled, "The Identification and Description of 
Exesplary Bilingual Education Prograns." 

V 

Again, thanJt you very nuch for the help you extended to the AIR 
, project staff. 

• # 
Sincerely, 



B-5 

An Equ«I Or>portufvity Emptcytr 



PROGRAM # 
COMPARISON OROV?: 



MEASl'RSS: 



SAMPLI.VG (Accritlon, conditior^s of exclusion, size;: 



LEVELS : 



STATISTICS: 



SICNIFICAKCE (Statistical, educational): 



Decision: Yes^-. So 
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APPENDIX D 



1. FORM FOR SUBMITTING MATERIALS TO 
THE DISSEMINATION REVIEW PAHEL 

2. PROGRAM SUMMARIES SUBMITTED TO 
THE DISSEMINATION REVIEW PANEL 

• Madavaska, Maine 

• Santa Fe, New Mexico 

• Phlla4elphla, Pennsylvania 

• Alice, Texas 

• Corpus Chrlscl, Texas 

• Houston, Texas 

• Klngsvllle, Texas 



« 



4 




c 



FORM FC«'SUBMinii« KATERIALS 
TO THE DISSEMIKATIOS .REVIEW PAKEL 

PROGRAM AREA (e.g., c«reer educ«tioa, dit4dvAnt«ged, reAding, Title III): 



I PROJECT TITLE : 

II LOCATION: 

III SOURCE AXD LEVEL OF ?VKDI>iG : 

IV PROGRAM START DATE : 

V BRIE? DESCRIPTION OF PROJECT : 

Goalt and objecXlves 

Context (ccxzaunity, school, student characteristics) 
Prograa description (grade level(s), years of operation, 

size, curricula, naterials, staffing, facilities, tlae 

involved, parental involvcaent, preservice/inservice 

training, etc.) 
Cotct (total, per pupil, initial Inplcaentation, ongoing 

naintenance, etc.) 

VT EVIDENCE OF EFFECTIVENESS (in a page or tvo) : 

Sussajirize fn appropriate detail the evaLuation evidence for the effective 
nets of the program or nodeL, in question. In order to be acceptable, the 
evaluation need not be a strict expcriaental design (i.e., longitudinal 
□easures, randoa assignaent to treatment and control groups, etc.), 
although this type of evaluation evidente would be the nost desirable. 
However, in order for the Office of Education to ^off icially recocza en d , 
a particular educational project, technique, or nodel for wide scale 
adoption, there oust be soae kind of high quality, objective, nethodologi 
cally sound, quantitative assessaent which deaonstrates that the project 
in question is effective and superior to other nore cocsaonly xised 
approaches or methods. Thus, in order to approve the dissenination of 
any project, the Panel will require a detailed sussaary of the relevant 

evidence including such tilings as: 

* 

— who conducted the evaluation; 

— sample sizes; ^ 

— improvements or gains in whatever outcome measures were 
employed; 

— the statistical reliability and educational significance * ^ 
of these Improvements; 

— sotae evidence that the improvements can be attributed to 
the program and are not just "normal" or "natural" gains 
(i.e., control group or norm comparisons, or some estimate 
of what would have occurred in the absence of the program). 
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PROGRAM AREA : 

Bilingual Education (French/English) -r 
PROJECT' TITLE : 

St. John Valley Bilingual Education Project 
LOCATIOH: 

Three coopefatiag school districts in Northeastern Maine, serving 
the towns of Madawaska, Ke.egan, rrenchville, St. Agatha, and Van Buren. 

<?CVRCES AM) LEVKT OF FUNPIKG: 

rne folk)ving figures pertain to the fourth year of progran operation. 

Title VII funds $175,085 
PROCRAM START PaTE : 

1970 

BRIEF DESCRIPTION OF PROJECT : 

Goals and objectives . Tne basic philosophy ot the bilingual progran 
is that it exists to suppleoent the existing progifan and to assist the 
cosraunicy in aeeting their linguistic, cultural, Social and edbnonic 
needs for bilingual skills. Designed specifically for this rural 
area where both French and English are aajor languages, the progran • 
is not viewed prinarily a« a language .progran, but contains language 
only as a oediun of the cultural, social,' and economic life the students 
will encounter. 

Context' . The connvnities served by the progran are snail towns 
in a rural area where the najor activity is potato faming. Kadawaska, 
the largest, also has a paper aill, which processes wood pulp punped 
across the river (also the U.S. -Canadian border) from its larger- 
neighbor, Ednundston, Canada. Before the boundaries were drawn this 
.was the area settled by the French-speaking Acadians. There is nuch 
connerce, social interaction, and interaarriage between the Canadian 
&nd American conmunities, and the Acadian .heritage renains a vital iti- 
fluence in the area. Ednundston is prinarily French-speaking, and nany 
pepple m the American towns afonfe the other side of the river also stilK 
epeak French at hone and use both languages in their daily life. Some 
children cone to school speaking French better than English, and poorly 
prepared for school in either language. 
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?rogr«a detcription , * 

r *i ' 

Grtde levela. years of operttion» sizfe — Duringr its fourth 
yetr of -operation, the prograa served 768 children iv. grades 

Staffing — The progran staff requi-red for this nusaber of 
participants consists of the following: one full-tiac- Project Director, 
one full-tine Curriculuo Coordinator, one -part-time ^rench Consultant, 
one full-tine Evaluator, 36 full-tine teachers, and 22 full-tiac teacher 
aides. All but a fev are bilingual. 

Curricula^ material, tine involved — Designed as a supple- 
pent to the regular curriculum, the bilingual progran 4dded a French 
'element which was completely new, and' also added soae new elements to 
the English curriculum, primarily a greater emphasis on spcalcing and 
vocabulary building. Thus, children now have certain basic skills ind 
concept© of the regular curriculum presented to them through the-nediua 
of two languages, plus a special series of topics in French'which relate 
to their cultural background. ^ 

Since the program operates in three districts whose curricula, 
staffing patterns, and physical arrangements were indfpendchjf , there 
are differences in the approach among the various schools. Grouping 
patterns may differ and English curriculum materials and activities 
may differ. However, all bilingual program clas^roooa share two basic 
and essential characteristics: They use a sequence of instructional 
objectives developed and refined by the teachers for each grade level 
over the years of program operation. They use standard procedures for 
evaluating achievement of these objective's. The ^valuator visits all 
classrooms frequently to guide and consult with the teachers in assessing 
student ^chieveme^it; this helps to maintain uniformity acfoss schools. 
The feedback from teachers and 'their involvement and responsibility 
for the curriculum and objectives, are important char^'t eristics of the 
program. • ' 

In addition to the uniform' objectives and evaluation procedures, 
the program has developed a special series of Fretrth curriculum materials; 
although not mandatory, the^ appear to be used in all classrooms. 
Teachers ^get a* folder -for each objective which contains the section of 
the curriculum guide for that objective. They generally build on thi.8, 
adding their own materfals aad activities. The French curriculum is 
built on the fact that the students live in a stable, rural area, with 
limits on experiential situations. The cotanercial materials used in 
Che regular curriculum seemed to have shortcomings in teaching certain 
things because they referred to situations totally unfamiliar to the 
students. The French curriculum is used partly to remedy this. If the 
students* test results show that -they are weak in a certain basic area 
of math, for example, that topic will be stressed the next year in the 
French component. For thieT reason, some objTectives'were revised 
ifnnually during the first four years, on 'the basis of test results? 
The content areas addressed through French instruction include mathe- 
matics, fflusic^ art, social studies, and language skills. About 

D-4 



80 



« 



two-thirds of the instruction cakes plate -lii English, one-third m 
French. ^ . 

The French cumculua aaterials, based on* the students' 
"cultural context", were planned curing the fir.st year of* the , 
progras to follow a logical sequence over the elenentary years. 
Expanding outward fron the student hiiuclf, the topics "ire as follows: 

^' Grades X-1 — the cM^<;, his facily ap.d friends ^ 

Grade 2 — the tbvn anS local area 

Grade 3 — the State of Maine and the Canadiax^ Province 

. of New Briinawick across the river 

% Grade ^ New England, Canada, and Louisiana (all 
hones of the Acadians) 

Grade 5 — the continent of North- Aaerica , 

Materials for Grade 6, presently being developed, will reflect a 
nusan developnent eaphasis, focuilng on the various parts of the 
vorld where French is spoken. ^ 

Facilities — The program can operate ih a regular' classroom 
without any special oodif icacion. The arrangeaents actually used 
.vary fron open classrooms created by knocking out old walls to self- 
contained classrdons yith desks in rows. 

Parental involvgaent — Parental involvement in the program 
has grown over the years, as parents, some initially fearful that their 
children would not receive a str6ng background in English, observed that 
the English curriculum was not being weakened and liked the introduction 
of activities m French which brought important features of local life 
into the children's schooling. 

The parents* organization has played an active role in getting 
the corsnunity involved in the program. They carried out community 
surveys of parent attitudes, and they solicited and contributed articles 
by parents for the project bulletin issued eight times -d year. Parents 
have also helped with hosting visitors to the program and have served 
on an occasional basis as volunteers in the classrooms. 

Preservice/inservlce training— Training has been conducted 
for teachers and aides each year since the program began. , Workshops 
have been offered by visiting consultants from universities in the 
,U.S. and Canada, by members of the Title VII program staff, by staff 
members from other bilingual programs, and by publishers of materials 
used in the program. These sessions, although optional and held 
outside of regular school hours, have had near 100 percent attendance 
by both teachers and aides since the program began. The inservlce 
'program has also included summer institutes, courses offered through 
the University of Maine at Fort Kent, visits ^to other systems and to 
other schools within the program^ and attendance at national confer- 
ences. The aides have participated in training sessions together with 
the teachers, and through a special study program at the university thty 
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earn credit tovard a degree in education. At<cndtnce at the Univcraity 
cour^tf is also voluntary » and an average of 20 to 25 CftXChera and aides 
have participated in each course. 

Sutrjects of the training seationa have includtd clattrooa 
canageaent couraes where the aides and teachers define their rolts 
working together in claesrooa situations. Other topics have been 
Acadian history, preparing objectives, ntw'experliaents and treads in 
bilingual education, use ot specific cattriais, evaluation procedures, 
developing English language arts activities, and infor&ation brought 
back by staff nenbers who^participated in conferences or visits.. 

Cost — During Year *1 the Title VII progran received funds 
totaling $150*000; AOO students vere served. Major start'-up costs 
included .$26,000 for Instructional materials and equipnent; once 
classrooms are equipped It Is assumed that only a cdnor expense for 
-new Mtrrlals '^lll be required for sone^yeaxs afterward. For 1973-74 



the cos; was $175,085 to serve 768^cHridren; The $228 per-T 
coat can be compared with S375 per pupil in Year 1. Regular district 
per-pupil costs for the three districts were $495, $465, and $567 
in 1970-71 and, respectively, $817, $637, and $727 in 1973-7A. One 
reason for the decrease ^.n ?er-pupll cost of the bilingual progran is 
that although the number of children served has increased, the 
program administrative staff has not. ^ Also, anticipating the end 
of Title VII funding at the end of Year 5, the local. districts have . 
begun a move to continue the program by gradually picki&g up the 
salaries of the aides. The Title VII expense for aides has reduce<i only 
somewhat, but miny more aides are now employed. The major costs to the 
districts fdr continuing fhe program without outside funding would be 
the salaries of aides and administrative staff, and the price of 
instructional materials. ^ • 

I 



D-6 



8':J 



.ERIC 



mOSKCE OF EFFECTIYQurSS: ' ' \ ' • ^ " ' . '* ' 

Evtlmtion. , ^ , • , , 

• ' * ' -•,.*■. 

An uauaually high 8tan<iaf<i of cv«lu*tioa- *rtG reportl&g -w&i a«t froQ * 
the first year of the prograo In 1970-?i by Heurlstici Inc. and Hr.. Ctlcan 
Hcbcrt jointly, and saiataincd since then by Cilsan-Hcbert, Alio unu«U4l' 
was the jjtct that ^tht services of the evaXuecote . . . wap cngftge^ Jbefo;re - 
tw st^fcft of *cla8srooa instruction; therefore evaluatloc of the^projept *is' 
boih fon:ative and svrrrrntive Ux nature." While ve do riOt always agr^'e with 
the&e evaluator« on the releva;ice of a fev ot ch^ir dcach^tratioos («.g«* 
we fict little store on the r6w «core grains froa pre- to post tests) their ' 
statistical analyses are always straightforward, appropriate-, and fully*, 
dlscussttd. The.oost recent report (1973-197A> was produced by Gilaan 
riebcrt, wjch the , technical assistance of Heuristics. / * ' 



Reported *re the result?* of 8tan4ardix<d tests, and the atcainaeat 
of product and process objectives, froa kindergai-cen to grade 4. A ipar- 
ticularly usefui feature in the aost recent report is a tAble showing 
longitudinal data for 1970-197^ for Total Mathematics and"- Total Reading 
for grades X t<? 4}'and the graphical presentations of distributions of 
stanines for each sxandardized subtest for each grade represent an addi- 
tional refinspent'. 

t • 

Each report has, for each section, a chapter on ^'Gosaendations and 
Rec05aendatlT3ns" which reflect shortcotilngs nQt^d (as well as successes) 
with suggested a'ctlos for the next year. - • a 

An unusual feature of" this progran is* that it^coyers three distinct 
school districts with vJhole-hearted collaboration, and a single evaluation 
report covering each district separately, as well as , their coobined results. 



Sapling 



. Different procedures were adopted for selection of progracT students . 
In Frenchvillc "a^nS St. Agatha (S.A.D,, #33) all students in kindergarten of 
Dr. Levesque School were included in the Title VII prograa. In Madawaska 
itsejLf , only thoss' students entering Jcindergarcen who' were volunteered by 
their parents were included; and in the third district of ^an Burden (S.A.D. 
#24) entering students were randonly asiigned to classes, and classes were 
* then directeji to pro^ftSsn or non-pt^ograa." Language doainance was not con- 
•eidered in allo'caticn,' so that French-dTJainant, Er^lish-doainan't and bilingua 
students w^re retained in successive grades, with only ninor^changes either 
way at the requ<5SC^of parents.' ^ , " * « * 

Six schools, two from each district were involved; ultimately 60 percent 
of all students werfe particij^ants; there were 768 .students , 36 teachers and 
22 t;eacher^aides in this last year of reporting. The table below is taken 
froa the report. The last row in the table shows ranges. In percentages, 
of incoaplet6 data for all tests. 



/ 



Eqrollaeht irt SillnsAai Pr&gr«» in l<i73-1^7i, by Schooled by Zr^Ct 



School Distrlcc, 









Grade 


(Trade' 

2' 


Grade 


^radc 

4 


Ail , 
Graded 


















Trenchville «nd 
















St. >Ag*ch* * 
















(SAD #33) 


19 




56' 




55 


60 


245 


















Mad«wA8kfi~; ~' " , 


43 




. 65-^ ' 


■ 75 


'74 


43 . 


300 


















Vao Buren 














(jSAD #24) 


46 




i 


50 ' 


3a./ 


43 


223 


Total 


108 




1^7 


I8a 








Attr;.tlon X 


7 




11-14 


' 14718 


7-10 


' _ * 1^-13 


10-13 



It should be noxtd that these procedures do not assuae randos allocation 
to'prograa o)r non-prograo* Coaparlsons are aade vlth national noma, and no 
bther coaparison or control ^roup was identified itt the report, but the ^ » 
superintendent* 8 offices of the three^school districts w6re able to give us 
addicldftal data, vhic^ a^lovecf us to cake sose cocparisons vlth the oon* «^ 
prog^aa students. This'vlll-be d^alt vlth later. * 

Measures ' 

The nev (1970) vers-ions of the Metropolitan -^chieveoent Tests have been 
used since start the program* for all grades except K,,vlth all subtest 
scores, beiSg^^iv(en>"/5^s^vftre giverf vlth Fom F as prc-test In Septeobcr, 
yfth the altamatf, Ffrta 9 /a* ^. posttest in May, i.e., vlth ievea Donths be- 
*cvean £he-tvo,* Levels used^ere J*rinary I fou'grade 1, Priaary II for grade 
2, EleaeaDary for grads8'3-^fod 4. Specifically, progress in Reading, Unguage 
and Matheaatlcs ^vere report^. 

lU grade i only, the Cotsaon Concepts Foreign Unguage foms I and 11 Test 
in French vas given as pre- and posttest*, vlth their own. local tests for other 
gradas, and vlth j^hiV»veaent ofv^peclHc ol>jective8- alao being recorded for ■ 
each student at ^^i^y grade levjcTi. 



The Durrell Listening Reading Test is a1«> k^vcn gra^s 2 aoA ^ . 
in jAnu«i7. The fests are applied throughout the HadtwasVu school 
district and Frenchvilie. 

Co»a^rison Procedures ^ ' 

T^t evslx^tors prepsred to sake three klRds of cocparisoas: 

* gains fro5 pr^- co post-test, 

* vith aaci«5\al nonss, ^ 

withcfjrcset criteria oT objactives. ' ' 

Understandsbly they avoided the possiViiity of comparisons bacvtaa schodU 
or school districts at?d reported fhelr ^ats ior total sasples by grade. 
Although other standardised ^ests vere spplied in the schools, only the 
Metropolitan was routinely applied in sll districts. The nain coaparisoas 
were therefore restricted to this instrument, and to specially constructed 
local t^sts and objectives. The latter of course could have no vlder 
Implications than local. 



For kindergarten, the .HA7 Prlner Level yielded standard scores and sta- 
"tilnes. Distributions on the latter were graphically plotted against a back- 
ground of s norcal distribution in addition to a table. 

For the higher grades, a table for each grade shoved, for eAch subtest, 
the size of the*sa=:ple, ceans.and stAadard deviations for pre- and post-tests, 
using both standard scores and grade equivalents. In addition for each subtest, 
the stanine distributions vere shown graphically superlsposed upon a nomal 
distribution. 

Oral, ability in french was tested at grade 1 with the Cossaon Concepts^ 
Foreign Langusge Test, French Version»with gains froa Forn I as pretest, to 
Fom II as posttest. Then th« caaplA jjaig^ dtvtdgd into Engliah-doainant and 
all others. Using raw scores, the evaluators aade cotmsrisons over the 
four years of existence of tke progran. 

As oentioned earlier a single table showed the trends and cohort perfonaances 
on the MAT for Total Reading and for Total Katheaatics, across grades for the 
duration of the four years of the project. The results were given in all three 
ceasures. Standard Scores, Scanines, and Grade equivslents. 

For their specif ied, criteria' in English and French, tables showing per- 
centages of ^students attaining all the objectives were given for each grade. 

Lastly, we did* redeive soae data on which we could compare perfor- 
mances of progran and non-prograo'students, for SRA tests taainly. 

Results 

- The oost revealing table is probably this extract froa their longitudinal 
study. ' , 



Table 2 

Results Is Stctnir.es and Grade Equlvalencs oa the 
Metrofiollt&n Achleveaeat Tests, for Total Reading and 
Total H*:^beseclcs Grade Level aad by Acadeslc Year 



C rytdt 



Subject _L^2p--21L 1971-1972 1972-1973 1973-1974 



Reading 
Mathexctlcs 



Sc*n 1 ne GE Stanlse GZ Stanlne GZ Staalae GE 



1.8 



Reading 
MatnesAtlcs^ 



1.8 
1.9 



i. <5 

1 . 8 



2.0 
l.S 



1.9 
1.9 



2.8 



3.8- 



Reading 
Mathe:iatlCs 

Reading 
Hatnezat ic6 



2.6 
2.5 



• 5 

6 



2.8 
3.0 

3.1 
3.8 



3.1 

3.3 

3.5 
i.2 



^.8 



Reading 
Hathenatics 



4.3 
4.7 



Ccaparing achievesent w^th placeaeat, there aeesa to be a flight 
loa« in grade 4; but for the reat, in 1973-74 average ^rade equivalents 
are cloic to, or even above placenent. , | 

fpllowing the transit diagonally, of thc^* 1970-1971 gradt 1 ^roup, it 
vH I be aeen that they started at par; lose tw.o Dontha in Reading and three 
in Math in grade 2 (1971-1972); lose an additional five oontns in Reading but 
caught up in KatheDatics in grade 3 (1972-1973); and finally! lost no Dore. 
ground in Read itig' and only a trifle in Katheaatics in grade (1973-1974). 

The next cohort did berter. In grade 1, {1971-1972) tj|«y were placed f 
as they had achieved in both subjects; in grado 2 tbey were i if anything a 
shade ahead in Mathexatics; in grade 3 they h'ad lost trifle in Reading, but 
were four nonlhs ahead on Matnematics. 

The tljird cohort (grade 1 in 1972-1973), starting again at par, were 
ahead on both subjects by the n«xt year (grade 2 in 1973-1974). 

There is therefore a suggestion that the prograa nay be refining its 
techniques and iaprovitig its achieveaents over tine. Also, at entry the 
average ability for the program is if anything a little above the average 
ability of the population (stanines of 6 to 7); thereafter the- average per- 
fomance in Reading is a shade below tills proQise (stanine^generally at flve^ 
with one four and two sixes), with Mathcnatics a bit^. betted (stanines of four, 
, several flvet, a t-ix, and one seven). When it is reaenbered that these chil- 
dren are for the laost part F rench-doninant , this seeas a good perforoanc«, ^nd 



D-10 



86 



ERIC 



Isprovlng over ti&4. In addition, they are of course taking an extrA sub- 
ject - French. * 

The evaluators for this prograc f clicked the coc=»n practice of teatia? 
the significa«:e of gains fron pretest to posttest; they did so using the 
correct t statiatic for correlated aeans and aa uiual theie differences in 
every caaa proved to be highly significant. As is our usual j)ractice, we 
discount the value of auch demonstrations. 

On the* other hand a oost useful device has been their graphical presen- 
tati on of diat ribution-of stanine scores against the backgrouBd of the nomal 
.^ii«frlbu t ion .\This was done for every subtest for each grade. Alaost uni- 
formly these shovTSiight upwarG> shifts, with a suggestion of peaking, or 
narrowing of the scatter, and occasionally with a hint of negative skewing 
as if the ceiling of the tests was saking its presence felt. 

For every grade, in every subject (including Art, Music and Social 
Science) objectives had been defined an<3 for each the percentage of students 
attaining this was given. ^ Tr.erc were about 140 of these objectives, spread 
fairly evenly over the five grades. For ^ninety of these, the percentage of 
students attaining the objective was ^0 percent or higher. For kindergarten, 
ona value was belov 70; only 38 percent satisfied the criterion on phonetl^cs 
-in French. The evaluators were less satlafied with the prograa's effects in 
gradaa 3^ -and 4 where about "a quarter of the percentages were below 50, aost 
of thez for Music and Social Science. 

Lastly, at our request during our site visit, we were given additional 
data fpr Kadawaska, which all<we,d us to =:ake soae conparisons for the Title 
VII grou^ with the rest of All non-prograa students.. 

in two schools in Madawaska, values of IQ were obtained at the 
grade 3 levcl'in Sovenber 1973 and in Sovecber 1974. The table below shows 
, the results. 

Table 3, 





10 MftAna and Standard Deviations.J(app 


rox. ) in Grade 3 


for 




. Title VII an<f Regular 


Classes for 


1973 and 1974 






Year 


Title 
VII . 


Regular 
Classes 


Total 
Group 




1973 Mean* 


♦ 111 




107 






13 


15 


14 




. N 


69 


58 


127 




'1974 Mean* j 


110 


102 


106 




SB*(±) 


17 


17 


17 




K 


76 


^2 


118 



♦Theye values vere derived fron quArtiles and medians, and are therefore 
approxixiate. . > * . 
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These differences are statistically slgaiFlcant at Ahout the 1 percent 
level, but of course could reflect, it least ia pju-t, the effect of cvo yetrs 
In the progr«3 for the-TItle VII students. However It is i^ortant to note 
that, especially in 1974, the standffrd deviations are if anything larger than 
usual; had there been explicit selection, we would expect all SD's to he re- 
duced -to well below 15 to about 9-. The fact that the SD for the toral group 
is abou; the saae as that for each of the aubgroups shows that there aust be 
considerable overlap between prograa and non-prograa distributions. • Kore 
than half of all students are in the prograa. 

Also for Kadauasica, we obtained the results of SKA tests applied in 
October 1974 to both Title VII and regular classes, in grades 2, 3 and 4... ^ 
The results are shown below. 

Table 4 



SRA Grade Equivalents for Title VII and Regular Closes, 





Grades 2, 


3 and 4 








Placesent 


Subject 


Title 
Mean* 


VII- 
SD* 


Regular Classes 
Mean* , SD* 


2.9 


Reading 
Language Arts 
Kath , 


2.4 

2.i • 
2.6 


1.2 
1.2 
0.7 


1.9 
2.0 
2.1 


, 0.6 
S.4' 
^0.7 


3.9 


Keadlng 
Language Arts 
Math 


t 

3.7 
3.9 
3,4 


\ 1-0 
\l-3 


3.0 
3.1 


1.0 
0.7 
0,7 


4.9 


Reading 
Language Arts 
Hath- 


4.7 
5.0 

4.6 ' 


0.9 ^ 


' 4.0 

4.3 
V A.l 


1.0 
1.2 
1.0 



♦These values have been derived froa quar tiles and-t3edians\ and are approxinate 



Here again we can see that the Title VII students are sooetlces a tififle 
behind grade placement in their average achievement, though tlWe are no clia- 
ulative losses. They are far ahead of the Regular Classes in eVery case. ' 
It is not possible to say for sure whether this is due to initial^ biases as 
a result of the self-selection (parents* choice), or whether it re!^lects a 
superiority accumulated as a result of two i>r- more years 4n the Titi^ VII pro- 
graa. However there is* one phenomenon that, should be taken note of: \ While 
the standard 'deviotion of grade equivalents for the regular classes is\closV 
to our expectations "(usually about '2/3 of a year in grade 2» rising to 
one ycnr at grade 4), Che values for Che Title VII are higher. This is con- 
trary CO what we would find U sele^ttion had taken place, but quite in keeping 
with the hypothesis that the treatment has expanded the ranges upwards. There 
is certainly no evidence of handicap for these, students, and they axe learnln 
French. 




fk.Cospari^oos beCveenCicIe V]I and Non-Ticle VII puTtlt vtt also &4de 
po«tlbIe tn the other cvo discrlct^ by additional dact< tbeas.ve have 

gradt -equivalencs, but ao standard deviations. 

School district Van -Suren used the Ke'tropolitan series of tests'* through- 
out the cchools. Besides Title VII, they also have the Follov-through progras^* 
but vere able aupply the folloving data for these subgroupings jtad for two ^* 
gradet» , ' 

Teble 5 s ' - ' . " 

>lean Grade Equivalents on HKI Grades 1 and 2 
nrlth Subgroiiping>> .for Hay 1974 * 




Gradt 




rule 


VII 




Kcm-Title VXI 






foilov-i - 






Follow- 






^ Placeatnt 


Subject 


thVi^u^h (N) 






through <K) 


KFT 


m 




Reading ' 


i.^' (22) 


2.0 


(21) 


• 1.3 (42) 


ii5 


(18) 




Kathezacict 


l-b (22) 




(2i) 


1.2 (42) 


1.3 


(18) 


2.3 


Read ing 


2.7 (21; 


3.5 


(12) 


1.9 (46) 


2.2 


(37) 




Hathesatics 


2.9 (21) 


3.3 


(12) 


1.9 (46) 


2.0 


(37) 



The si;periority of the Title VH students is patent; the lover of their 
Subgroups (Foilov-through) have higher perfqraances than. the better of the 
Non'-Title VII subgroups (Non-follow-through) . *• 

"For FrenchviX'le we have $RA test results for grades 2, 3 and 4. In 
gpsde 4^ by specif request froa a teacher with a group of 20 low abili^ty 
stud^t8» her claA was included with 22 other students in the Title VII 
prograa; thii of course depressed the nean performances given below. - 

Table 6 

^SRA Mean CE. 's for Title VII and Non-Titlt VII Groups 



Grade Placenent 


Subject 


Title VII 




Non-Title 


Vll (K) 


2.9 


Reading 


2.4 


(70) 


1.8 


(47) 




Kathesiatlcs 


2.4 


(70) 


2.1 


(49) 


3.9 


Reading 


3.6 


(73) 


3.2 






iiathemati^s 




(73) 


3.1 


(59) 


4.9 


Reading 


3.7 


(42) 


4.6 


(88) 




Mathezaatics 


5.6 


(42) . 




(87) 


It should be 


reaexabered that grade 


4 included 


a special class of 


• 

low- 



ability students. For the other grades the title VII students are ahead. 
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CoQpents — ^ 

The iapression created, after the site vlaic, was of thorcftighn^sa of 
both the educational and the evaluation procedures. It vaa a pleasure to 
find no trace of political or ulterior taotivation, but on the contrary a 
clear deteralnation^to avoid handicap for their students in English, without 
losing their historical and predoninantly French origins and culture. Every 
"classrooo had evidence of considerable esphasls on conceptual development 
in both languages, and we frequently heard the class switch frco French to 
English to reinforce sooe non« language these, as for exasple in social studies 
or sathesaxics. ^ * * 

The data snd statistics are acre solid than startling^ and thtf nore 
believable for that. They are quite. in keeping with what ve saw and with 
the cocpetent professional atoosphere we found. 
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PROCRAM AREA : 

Bilingual Education^ (Spanish/English) 
PROJECT TITLE ; , - ^ 

ESEA Title VII Silingjjaal Education Prograa 
LOCATIOK: , " 

Santa ?e, Kev Mexico 

V 

SOURCES AKD LEVEL 0? RTOING : 

The following figures , pertain to the fourth year of prograa operations 
and do not include the local funds noraally allocated by the Santa Fe Public 
Schools for the education of the children vho are served by the bilingual- 
prograo. 

Title VII Funds $ 72,281 

Sttce Funds 58,615 

Total Year U $130,896 
PROGRAM START DAT& : 

1970 > ' • * \ ^ 

BRIEF DESCRIPTION OF PROJECT ; 

~ ♦ 

Goals and objectives . The prir.ary goal of the prograa is to develop 
children with a good self-concftpt who can cosaunicate and learn in both lan- 
guages, English and Spanish. Part of the bilingual philosophy is to maintain 
the language and culture of the Spanish heritage In t^fe^Southwest; at 'the aaoe 
tine; giving children' a good education in EngUsh is viewed as essential to 
the role of the schools. r 

Concext. The Santa Fe area still reflects strongly the traditions of 
Its earliest- inhabitants, the Indians and the Spaniards. Within the city 
itself, tl>ere is a considerable second-language influence of Spanish. Accur- 
ate measureaent of the , predominance of Spanish has been difficult, although 
" the Santa Fe Public Schools ^ave made various attempts. The actual proportion 
of students whose primary or home language is Spanish probably falls somewhere 
between the 63% based on Spanish surnames^and the ZUX reported by ^ survey^-of 
parents.*^ The three Title VXI ^£hools average SCR; Spanish surname; all three 
arc also Title I schools. • ' 



ogram description . *v , 

Grade levels, years of operation, size — During its fourth year of 
operation, 1973-74, the program served 419 students in 19 classrOcfas. Of , 
these, 8 were first-f,rade classes, 5 were second grade, 3 were- third grade, * 
and*^ were fourth grade. Twelve of the classes, one at each grade level in 
each of the three original Title V|I schools, were included in the evaluation. 
The .other 7 classes ^j'ere part of a horizontal expansion into ney schools. 
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- St»ff Ing— The prograa staff reoulted for chls nusber of i>artlclptncs 
consists of Che following: ' 1 full-clae projeqc director, 1 full-ctoe aatert«ls 
speclallic, 1 full-clae sccrectry, 1 part-cloe evaluacor, 19 full-tiae teachers, 
and 19 ful^-clae aides. All Instructional personnel are blllQgual. 

Curricula, aaterlal, tltae Involved — The children served by the Santa 
F« BlUngual Program are predonlnentlv Spanlsh-surnaaed, but very rarely Spanish 
^nollnguals. Generally, they coae to school speaking both Spanish anc English 
rather poorly, ^wlth Spanish the prlaary language for at least a third of then. 
The bilingual prograa adds to, the regular English prograa a Spanish Instruc- 
tional coaponent which coapleaents and reinforces the Instruction In all con- 
tent areas. Thus stydents receive a bilingual presentation of all the topics 
of study In the norsal currlculua. For e^caaple^ a teacher night Introduce a 
certain nath concept In Spanish In th^ nomlng, then reinforce the lesson In 
English that afternoon. In language arts, the regular stat^-candated English 
currlculua of the Santa Fe Public Schools Is followed. In addition, Spanish 
language Instruction Is provided, and parallels the English Instruction so that 
objectives are at roughly the "sase fttage In both. A child should generally be 
reading at about the saae level In ^oth English and Spanish, for exacple. 

The prograa usea Spanish currlculua natfcrlala froa the Spanish Gurrlculiio 
Developaent Center (SCDC) In Mlaal, and Is participating In the ^'regionalizing" 
of theae materials by suggesting revisions which taake thea oore appropriate for 
the Southwest. Four strands of the SCDC currlculua' are used: science-oAth, 
language arts, Spanish* and fine arts. The nattrlala Include belravloral objec- 
tives And crlterlop-referenced tests. In addition, the teachers have 
developed special social studies objectives which are locally'^neanlngftt^ Thus, 
although the Spanlah currlculua parallels the regular English currlculua. It 
alao adds topics' .which are especially relevant to the orograa children and their 
bockground. For Instance,^ the Spanish words and historical figures which gave 
nanes to Santa Fe stree^ts^are Included-Xa.the second-grade study of conaunltles. 

A bilingual' aaterlals center has been established to serve the 
teachers, providing tbea with SCDC and ojther Spanish-language niaterlal3,._and 
with sptclal suppleaentary materials In English. Th6 materials siietlallst 
consults with teachers about materials they need, and Is also In the process 
of bringing together the many materials developed by Individual teachers, which 
will be selected, catalogued, reproduced and made, available tx^ all program 
teachers. 

4 . * 

Classrooms are divided Into "Interest centers" where children work 
on the various subjects In the curriculum. These centertf are stocked with 
materlarli In both languages, and are equipped with*' various audiovisual and 
Independent atudy aids, such aa language mastera, listening center, typawrlter, 
and fllraatrlp viewer. For language^arta/readlng, pupils are grouped by ability; 
In^nJl other subject are/is groupings are flexible according to aacHr child's 
naeda ond intereats. Genially, the teacher worka with chlldi^an In one of the ' 
room's centers; the aide works with a second group, and the rest work Indepen- 
dently, alone or in small groups- The frequent shifting of groups and subjectg 
complements the shifting of the language of Instruction, as the teacher conducts 
a lesson with one group In Spanish, and they* progress to an Independent, activity 
In cither language, and then perhaps to an English lessen with the aide In 
another center. 
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Ficilicie8--7he progran can operate in a regular classrooa; the only 
aodifictcion required is Che rearrangeoent of furniture to create the different 
centera. 

Parental involveaent -- Ov*<'^r t!>r years, s very active Parent Advisory 
Cossalttee Haa taken the responsibility of organizing parents for a variety 
of activities in aupport of the progras. Parents have viaited the legislature 
to convay their approval of the prograo, have accompanied children on field ^ t 
trips into the comunity or the nearby pueblos, and have volunteered their help 
for special occasions in the classroom, displaying^ crafts and custona, and pre- 
paring foolJ. On one such occasion, parents- brought varioua types of traditional 
Lenten foods during Lent, and they, their children, and the teachers gathered 
in the gycmaaiua to share the neal. 

The Parent Advisory Cocmittee also coordinated special public events to 
acquaint the cotsaunity with the prograa. One of these was a series of neetings 
in which a paniel of experts' disx:ussed bilingual education, oeobers of the 
Bilingual Prograa staff talked about Santa Fe'^s program, and deoonstrations were 
presented by children and teachers. Xnother event was a citywide bilingual pro- 
gram cocaetaorating Cinco de Mayo, in which children from the progran presented 
traditional dances and celebrations on a Sunday afternoon. 

Preservice/inservice training — Teachers have had an important role in 
developing and refining the progran. They have written behavioral objectives 
each year aa succeasive grades were added, and have revised these In subse- 
quent years. Beginning with the second year of operation, when grkdes one 
and two were included in ^e program, inservice training for teachers and 
aides was offered through universities in the area. The district contracted 
with the University of New Mexico in 1971 and with New Mexico Highlands Univer- 
sity in 1972-75 for the provision of inservice training courses for which aca- 
demic credit would be given. The courses were carried on throughout the school 
year, and included such topics as development of behavioral objectives, assess- 
ment of entry levels, specific teaching techniques in the skills and concepts 
areas, using Spanish in teaching, naterials development in Spanish, Spanish 
culture in the Southwest, and a seminar and practicum in bilingual education 
focusing on components of the Santa Fe program. 

Cost — During Year.l the progran expended $58,598 and served 77 child- 
ren^-ln three first-grade classrcons; during Year 4 total program cost was 
$130,896 for 419 children, 19 classrooms, grades 1-4. These 19 classrooms 
indlude one class at each grade level in each of the three "pilot" schools 
where the program was first installed under Title VII; these are the classes 
included in the evaluation. In addition, parent requests prompted the hori- 
zontal expansion of the program into 7 more first- and second-gracfe classrooms 
in 5 other schools. Here, as in the" pilot schools, the program adds a grade 
each year. Start-up costs for each new classroom were primarily for materials 
and equipment and the aide's salary. Continuing costs have included the salaries 
of aides and 'of the program administrative staff, inservice training, evaluation, 
and general 'support, services. Because the expenses of the program administrative 
staff have remained relatively constant as the number of children served^ has In- 
creased, and because much of the equipmejit lasts for more than 4 years, the pro- 
grants per-pupil cost has decreased from $761. in Year 1 to $312 in 1973-7A. 
Thia i? in addition to the regular district per-pupil cost of $803 for 197J-7A. 
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In 1973-74, funds were allocated as follovs: Tide VIZ funds covered the 
third- and fourth-grade cldssroor> In the pilot schools. Grades 1 and 2 in 
both pilot and expansion schools ve're covered by state bilingual education 
funds. DGringCthe coaing year» the Santa Fe Public Schools plan to pick up 
part of the pro^raa cost as its expansion continues, an estimated $18,000 to 
pay for aides. 
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EVIDEKCE OF EFFECT I VEN'ESS: 



Evaluation . This was dont by Dr. Charles F. Leybra of California State 
University* Los Angeles, and submitted for the acadeaic years 1972-73 and 1973- 
74. The earliest reports for 1970-71 and 1971-72 were done by Dr. Kari-Luci 
Ulibarri. The Ex>tt recent report is the Dost conprehensive in teixa of d«t4 and 
analyse!. It is a substantial effort covering the year's perfonaances by grades 
l-4» for bilingual groups and theif control groups at each level. The- bulk of the 
report cmxilBts of the data for each and every pupil on every test given, vlth 
statistical susxnariesl 



Of these, sumiaries, results of criterion-referenced testing are fully 
reported ior both English and Span?sh, nainly in the fonn of proportion^ or 
— percentages, with some interesting consient by the evaluator. He also reported 
on the results of standardized tests using more refined statistics; these cover , 
only pages 7-12 but, supplenented by a little additional analysis by AIR, are 
the Dost revealing of the effect of the program. 

It should be noted that the earliest report by Dr. Mari-Luci Ulibarri threw 
light both on deficiencies in .experiniental design, and on shortcomings of the 
program; in both areas her report appears have had the" attention of the school 
district organizers with good resirlts in the next year. In particular, her 
coianents on the use of covariance analysis for adju8tinent& of 'differences in 
control and experimental groups, and on the need for random selection of assign^ 
nent, are models. While random assfgnnent to control and bilingual groups was. 
not done, the sample use4 for control was randomly selected from all pupils not 
placed in the bilingual group; this yielded some improvement for the comparisons 
which followed, which were in the i?ain» for each grade level, of performances by 
^ control and bilingual groups on each subtest of the Metropolitan Achievement 
Tests separately, for Total Reading and^ Total Mathematics, and for attainment 
of objectives in English and Spanish. ^ - / 

Sampling . Three schools were involved. In each, for each of' grades 1-4 
(New Mexico tjoes not have kindergarten) , all students in the 'bilingual Xi-C 
treatment) groups were used, while random samples of about 24 (stratified by 
achool) were drawn for each of* these grades from all students not placed in bilingual 
groups.. * . 

It shqulfi be noted that divls^ion into bilingual and non-bilingual groups 
was not random. Parents could elect to have their children placed in either 
group,., and to have them transferred at later stages.' This process is likely to 

- have caused biases though we made one attempt to check on £his, as w^l be seen 
later. One hypothesis we were not able to test was that "perhaps at this stage 
parents whose children were already fairly fluent in English wanted their child- 
ren to^learn more Spanish, while those whose children had difficulty with English 
preferred to concentrate on it. , While thia hypotliesis did not seem to liold in • 
the earliest stages of the pr^^ram, a recent phenomena, superiority in English 

* from the beginning of schooling, is difficult to -account for without it. In 

tfie control group for each of the grades between 21 and<5 students were initially 
placed, with attrition losses generally of four or less; grade 3 lost 5 out of 25. 
Each of the bilingual groups started with 65 or more, with a maximum attrition. 
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m grade 1, of 18Z; tn grtde 4 only 7>ut of 
The ttble below glvts fuller detttl» 


68 did noc 


eocplece all tests. 




J 


TABLE. I 






0 


t 


' Stnples Used in 1973-1974 , 








Bilingual 


Control' 




Grade 


Initial ^ Used 


Initial 


Used 


* 




65* 53 


'24 


24 - 




2 


66 58 


21 


18 






•22 62 


25 


20 / 




4 


^68 -61 


23 ' 


22 




TOTALS ^ 


271 234^ • 
' « 


93 


'84 
V— 





Losses were the result of Incoraplete data on either pretest or poattest, or 
both. . , 

» - ^ V, * 

Meajures. The Metropolitan Achieveoent ;feat forms F and H, approprmte 

levels, vece ^fBtd for pre- and. ppscteetii^* Criterion-referenced neaiures for 

Spanish and Sn^il^fths'^e *l80 te;rort^^ l3t5(i^th^^ form of "nunbers of IteM correct « 

^n each test." Au^titxteats for the MAtJare^eported separately', so that per- 

fomance in natheoatlcs csn^lso be ,evaluaDed. t 

I ^ . 

In addition, scores'on tlie Otls-Lftnnon Mental Ability lest are avaJd^ablp. 
This tes^t was applied to eacK-Pypll at entry to school. 

Comparison procedures . Form F- of the MAT was given as pretest to each of 
the subgroups during the fall (September) and form H was applied as posttestf 
'in spring of the following year (May). Thus, comparisons possible fox each gradfe 
separately are raw score means with standard deviations only: , 



Bilingual group with control group oVi pretest scores 
Bilingual group with central gfoup on posttest scores 
iilllngual group wlth^ control group on ^alns * 
For each group, gain from pretest to posttest 



The criterion-referenced teat& also yield comparisons of bilingual with control 
groups for each grade and on botK pre- end pgsttests. These results are ^Iven 
as ''percentages for each school and for bath languages. , • . 



With the lowftr overall 'performance It was* found to b6 better to extend ^the> 
use of each test level by half a ,year 

the posttest In order to avoid 
floor effect. ^ .« ' 



90 



'(since wc 4i¥coimt the valine of «uclf deaoa»tr*tlcm^) to" calculate the slsalfl- 
xAoce of dtffcreacM of poattcats. Oaly the fotal Kath ooattesrs differed 
algnlflcjtatly '<? < O^IZ); Wof the 16 teat dlf fercocea,-c«ly :flve yere signi- 
ficant ^«t the 5X level or better, and oray oae via negative* Thua the bilingual ^ 
groopa are at the leaat holding their own or better, vMle of courat getting 
inatructloa in Spaaiah in addition. The aupariority of the bUingual group over 
the Control at the ptcteaca froca- grade 2 owarda could" be the effect of treat- * 

sent received up* to that tlDc* ; — - , , , ' ' 

* - 

Attention is drawn ti> the Total Reading perforxances of the two grade 1 
.groups where for both ?re^ and postte^t the blHngual group had significantly 
higher acans. 7or the pretest^ at leasts this, is noteworthy sin935?;tbere is no 
earlier trfataent to account for the superiority (Hew Hexlt^o has-^C^ kindergarten). 
We vfcre provided wither Is-Lcnnon Dental ages fo^^iUi ^^^^^ ^ groups 

by Mr. »Roacro, and* foind these su=aaxy statistics: * 

Bilin^jal groj»P ■ Control group 
Means (Months 73.56 / 72.96- 

Standard Deviations ' 10.68 . " 8.42 

" Sasplc Sizjr ^ ^ 52 ' » . * * 

.' (Note: One pupil in the bilingual group had no test result" here) 

The difference of 0.6 oonths in favor of the bilingual group Is in the 
right direction, ^ut quite insignificant and aurely not Urge enough to account 
for the differences in Total Reading, which are of. the order of a half standard ^ 
deviation. Thla seeaa to point to ^ooe biaa in- the bilingual group; indeed It • 
aeesa reasonable for parent^ whose children are already fairly fluent in Engllah 
to.aeek the addition of Spanish instruction, and for thoga whoae children need it, 
t6 prefer additional attention to their laglish. .If thia hypotheais could be 
^austained, it is in any case a recent developaent; in the first two years of the 
-"prograa, -the control" group was consideraljly superior ;to the bilingnal group. 

The results bt^he' «iterlon-re/erenced tests will not be given in detail 
here. In t^e evaluation report, data is. given in the fom of percentage passing a 
test. Results are given for each of 12 to 17 tests, both, as pretests a^ posttcsts 
for bilingual and contrdl groups for each grade, in each of the three schools, -n 
each of^the two languages. Results varied -f roo 0 percent to 100 percent in alnost 
every subdivision, oaklng suraary difficult. However, in Table 4 we have t^ 
jiverages of percentages passing the 12 to 17 tests in each subgroup..- It is not • 
possible to condense the table further. ^ / ,J 
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* (since we dttcoimt the val^t of sxjch' dcacraitr*tlo^) to'caiculate the sisaifl- 
'CMce of dlf fereacet of postteits. Only the TotU K*th Roittesrs differed 
*lgaiac*atly <P < O.iX); W of the 1^ test differcxxes, coly five vcre sigai- 
ficftnt at the 5X level or better, «ad't»ly oae vi« negmtive* Tho* the bilingual ^ 
grocpa are at the least holding their own or better^ while of couraa getting 
inatruction in Spanish in addition. The auptriority of tha bUlngual group over 
the Control at the ptetescs froa* grade 2 onwards cduld be the effect of craat- ' 
oent received up'to that tine. , - , . . ' 

V 

Attention is drsva to the Total Reading perfonsances of the two grade 1 
-groups where for .both ?rc-^ and postcest the bUingual group had sU^i^icantly 
higher aeans. For the pretest^ at least, this, is noteworthy sin^^'xhere is no 
earlier trfAtaent to account for the superiority (Hew Mcxito has'lSO kindergarten). 
We wfere provided with^tis-Lcnnon Dental ages foj.^ chlldre^i in the two groups 
by Mr. ,Roacro, and foind these sussaxy statistics: 

Billnyn.ia1 zro^p Control group 

Means (Konth^ 73.56 72.9fr 

Spandard Deviations ' 10.68. ' 8.42 

' Sasple Sizjf ^ 7 ' / ■ 

. (Kote: One pupil in the bilingual group had no test result" here) 

The difference of 0.6 Donths in favor of the bilingual group ±3 in the 
right direction, ^ut quite insignificant and surely not large enough to account 
for the differences in Total Reading, which arc of, the order of a half standard ^ 
deviation. This seeas to point to «o=e bias in the bilingual group; ^^^f j;^ ' 
seass reasonable for parent^ whose <;hildren are already fairly fluent in English 
to/seelc the addition of Spanish instruction, and for thoaa whose children need it, 
t6 prefer additional attention to their English. -If this hypothesis could be 
-sustained, it is in any case a recent developaent; in the first two years of the 
-Vograa,-the control' group ^was consid6ral?ly superior ;to the bilingual group. 

The results bfTthe criterion-re/erenced tests will not be given in detaU 
here. In the evaluation report, data is given in the fom of percentage passing a 
test Results are givjen for each of 12 to 17 tests, both. as pretests and posttcsts 
for bilingual and control groups for each grade, in each of the three schools, ^ 
each of--the two languages. Results varied -froo 0 percent to 100 percent in alnost 
every subdivision, oaklng suraary difficult. However, In Table 4 we have 
averages of percentages passing the 12 to 17 tests in each subgroup. r It is not - 
possible to condense the table further. • ^ J 
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TA3L2 A .a 

Avcr&ges of Tocal Percent Correct In Criterion Tests* Pre ?ott» in English <nd 
Sptnlfh Versions for £«ch Crsde In Each .of the Schools Participating in 1973* 7A, 
for Control snd Bilingual Groups Separately (Derived froa 197S-7A Evaluation ReportJ 

H 



G2ADES ' 



One* Two Three 



- ^4 w tr ' 4 lr« Post Pre Post - Pre Post Pre Post . 
sagllsh Version 

' Alvord , 

^ Control Group " 55 76 30 , 61 31 47 46 58 

Bilingual Grpup -65 81/ 43 87 32 90 74 81* 
Larragoite 

Control Group / 71 89 38 , 43 20 35 • 55 63 

Bl lingual Group 81 37 48 75 30 ! 90 .57 73 

t 

Agua Fpla ^ . ^ ' 

Control Group - 73 90. 48 69. 24 36^ 49 56 

, Bilinguar Group ' 71 84 52 85 31 50 49 63 ^ 
.i . • . - • 

S\ . - / \ ' 

Spanish Version ^ , . 

Alvord ' , 

y Control Group- 37 54 18 ' 41 , 10 18 34 26 

Bilingual Group 42 81 3^ 89 16- 71 63 80. 

Larragoite 

Control Group 41 52 26 28 7 ' ^ 24 19 22 

Bilingual Group 46 69 ' 35 65 22 93 30 3? 

Agua Fria • r 

Control Group 31 47 30 37 4 17 13 ^5 

Bilingual Group 4Q 82 44 .85- 23- . 41 - 25 ,38 



J 



In the .first colira in Ttblc 4 arc the «ver*ges <ff the percentage of 
students passing the 12 criterion pretests in ^ch group In each school in grade 1. 
It should be noted thst once again the'biilngual group has better results thap. the * 
control group in every case but one* As this occurs before treatacnt, .it vould 
seea that the bilingual, group is oore select than the control group ^TRiis blaa 
cay be present in later* grades,- although differences betveea groups SKfcar to 
&creasc at^the posttest in English. At least the bilingual group appe^s to suffer 
no handicap' ixx^&iglish. In the^ Spanish versions* the bilingual groups gcjerally 
do aach better than the control groups. - " ' ' ^ 



In Table 5 Jbelow ve give the posttest grade equivalents for grades 2, 3, A for 
each of the two groups, 

' ' ' * -TABLE 5 - 

Grade^Eqoivalents Keans of Posttest (Metropolitan Achieveaent Tests) 



Toxal English Reading 



'Total Kathesatlca 



Grade Placement 


BilinKuai 


Control 


BilinKUtfl 


Control 


* 

1.8 


•(61) 


(i6) 


(59) 


<A1) 


2.8 


' 2.33 ' 


' 2.25 


2.66 


2.03 


3.8 


. 2.96 


2.95 


3.14 . 


2.74 


^.8 


3.92 


-3.52 


3.95 


3.87 



* Grade equivalent^ are not avail«b;ie for grade 1; percentile ranks are given 
instead. Table derived froa 1973-1974 Evaluation Report. 



We can draw thfese conclusions froo t'he last three tables and the reports 
on the criterionrteferenced tests: • ^ *' 

• Both groups ere -losing ground each year-^ coaaon finding. 

• ' " ^ ' .0 

• The bl Ungual group is ahe*d of ths control group at the posttest- , 
for every grade fpr both Total ReaJing and*Total Matheaatics. 

• The bilingual group incVeases its lead in Spanish every year. 

• If anything, the. superiority of the bilingual group is even nore^ 
pronbunced for nathcnatlcs than it is for reading. ^ . 
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* Coaaentt . We shoui<l have liVed h«ve inclosed longitudinal studies since 
Xhe ^rogr«s h** been in exiatcace since 1^70, ao chat the first grade 1 pupils 
are nov in grade 4, but it A^pold have involved us in further saaipolation of data 
with not enough cise. 

* Upon visiting thfe classes (and vc visited every clas^ in All* three schools), 
the satisfactorx cocparisons cade occa8ion:td no «urprl«e» All three schools set 
high standarda^of clcahliness, orderliness^ and of energetic and purposeful per- 
formance by all principals and teachers. All classes fros which the control 

, saspl^ were d/awn were examples *of traditional teaching at its best, with never 
aore than about 25 pupils per teacher, and with a good -deal of. evidence of both 
training and effort on the part of the teachers; but the bilingual groups bad in 
addition an aide in every cla<«sroos and all the oodem anclllaries of language 
casters, projectors and overhead projectors, tape recorders, etc. It would have 
been s surprising ^nd interesting phenoaenoqi if these things aade no difference. 
And on Ibe other hand, knowing that the billng^l. group had Spanish' as an addi- 
tional course of 8 tody, It would have been suspicious if the differences were 
larger than they were; that this did not happen is alaost certainly due to the 
careful experlaental design, which excluded cany squSFces of bias. 
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NOTE: Thti progrta is rccoosendcd for* cwo atjor re«OQS. ?lr«t, 
Che ^VAluACloa outconer show chtc aott objectives, Are being aec 
or turpAtstd. Second, the evaluecloQ dtslga Is extsplsry end — 
cehle. See ^eges 8 through I8> ' * ' 



PE0GRAM AREA: 

Billnguei Educetlon (Speaish/Engllsh) * « 
?20JICZ TITLE: • . ' / 

Lec*s Be Anigoff 
LOCATION: 

Philedelphle, Pcnnsylvenia • 

SOURCE- AKD LEVEL OF FUXDIKC: 

The foHovlng figure^ ere for che fifch year of operaclon. 

Title VII Funds . $695^066 

Other Federal Funds _ 265,0(Je 

Local Funds - 705,000 ' , 

Totel ^ter 5 Fun4»- §^,6A5,066 * 

PROGRAM START DATE; ' -t* 

BRIEF OESCRlPnON OF PROJECT: ^ ' ' ^ 
' Coals and Objectivea . The Let'*s5e Anlgos^rograa Is designed to , 



• Increase scude^s^ .first and second language coc^^tenclea , 

• provide for conceptual^developnent In the students* oo^hcr • 
tongue, * ■ I ^ 

• Increase students* self-estffecS, , , 
^ locate and/or develiop currlculun caterlals relevant to the 

• needs' of Philadelphia's Puerto Rican ^tu^etxcs, 

• identify and develop ^ paints o^ Spanish speaking cpaeBmity 
ncsiers, and ' ' 

• - sy^t'cnatlcally oonitor and iaprove upon previous prograa 

activities. ^ - ^ ^[ 

Context . Progran students attend four high schools, two junior high 
scltools, end five elei^tery schools located is the Inner area of a port 
of cntry'city. Of thc^,830 Spanish language doi^lnent studcots (melaly 
Puerto Rlcen) end the 1,000 English linguage doniti«t studeht# served by 
the progrea in X974, lOOX ceoe froo fanilles with 'ah everage annual income 
under $4,000. Progrea schools ere characterized by over-crowd ing, high 
ebsenteeisa, end a' 40X turnover in stud^^t body ea^h year. 
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?rograa Descrlprian ^ * 

rh^ Let's Be Aalgos Prograa consists of three educational coo- 
ponents: Model A, Model B, and Arriba. These cocsponea^s are distin- 
guished by the populations they server their staffing patterns for 

iiistructloaal and st2?>port personnel, and by tb^ tine they devote to 

each language. On the other hand. All three cooponeats '-have sin i lar , 
. cur r leu lira philosophies, parent-crccsxinicy involveaent, Inservlce- « 
preservlce trailing, and prograa s&onltoring. l\ the retal n der of this 
progras deicrlptlon, the aspects uhlch distinguish the three cccpoa- 
ents are described," for Model A,-«odei B, and Arriba in turn. Then 
tloilarltles aaong the xhree cooponents are.suaaarized under the head- ' 
lags Curriculum a^'Katerlala, laservlce-Preservlce Training, Parent- 
Cosaunity Involveaent, and Prograa Monitoring. 

MODEL A * 

Grade levels, years of operation, size — Model A be|ad la 1969 
in the newly constructed Potter-Thocas desonstratlon school t6t^ bilingual 
education. During the 1973-74 school year, all 760 Spanlah-dorainant 
students and 640 Engllsh-doalnant stirlcnts in grades pre-K through \ at 
the schorl were served. (One of xfe kindergarten classes Is an all -day 
kindergartc^ for nore advanced students.) - ' . ' 

Staffing (lastr<ictlonal and support") — j' In Model A a bilingual, 
Spanlsh-doalnant teacher and a bilingual, Ej^glish-dcolnant teacher . 
work together vto provide instruction for 70 students who are working 
at the saae Srade ^evel.i The Spanish-4oQlnant teacher teaches onlF^ 
in Spanish and is the hoberota teacher for the Spanlsh-<i<?a.lnant students. 
The EnglishTdoQlnant teacher teaches -only^ in English Snd is the hoae-^ - 
rooct teacher, for ?he Engllsh-doalnant students. ' Students usually 
stay. with their hooeroocr class when receiving Inatruction Ip language y/ 
art8| reading, -social studies^- -and nath€aaatl1:s. If certain studenSs 
seca ready to read 'in a second language before others in their class, 
they pay j(51n oorei^dvanc^d classes during the language arts period. • ^ ' 
Students froa the English-doalnant and the Spanlsh-doolnant hoaeroons 
oeet together, weekly ff or such a#nlpulatlve activities as science 
experia^nts, art projects, dancing, and singing. 



^ Non- teaching support perspnned ^t -the Pocter-Thcoas deaonstratlon . 
school" include two readlrfg specialists, one aatheaa'tlcs specialist, 
a bilingual librarian, a aedla sifeciailst, and a prograa coordinator. 

3e. suSject-^tter specialists plan deaonstration lessons, order new 
t^rials, help place stiiidents la the proper ^classes, and , supervise 
progreta teachers.- The prograa coordinator also serves as liaissn betweea=^ 
the bilingual prograa sta/f and the school principal, who is not bilingual. 

I 

Tiae Involved — In general, pre-klndergar^en and kindergarten 
stuUents in Model A re'creive a half-hour of instruction dally in their 
second language. Students in the all-day kindergarten clifsfry grade 1, 
and grade 2 receive 1 hour of instruction per day in their second language. 
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S^nlsh-doolnant students in gr^ide 3 receive 2-1/2 h9ur^ of their 
l4»truction in eich Unguagc* while EnglishrdooinAnt third ar*de 
students ^receive I-IM hours of instruction In Spsaish *nd 5-3/4 
hours of instruction in English. 

According to their linguistic ability* stttdents ,asy receive specisl 
help froD two "English as a'Second Language*' teachers, one **Spanish*as 
a Second Language" teacher, and "Two Spanish reading teachers. One full-; 
tlae aide and one part-tiae aide are also available to help each teas 
of teachers provide Qor^ individualized instruction. 

MODEL B 

Grade levels, ye^rs of ope^tiOn> size — Model B began in 1970 
as a replication of Rodel A in other schools. During the 1973-74 
school year» Hodel B served i'SO^fSpanish-doainaiit and 80 English- 
dcQinanJ: students in grades 1 through 6 at Ludlov. Elesentaty School, 
as veil 'as 100 Spaniah<-4oQlnanr and 50 English^ddbinant students in 
grades 1 througK 3 at Miller Elenentary School. Stjudents, parents, or 
school officl^li aust re^iuest that a student 'participate in the Model B 
progysB, aa thore Is only one Hodel B class .per grade level at each 
school, ' . 

I 

Staffing (Instructional and support) — Teachers in the Model B 
ptogran are bilingual Spanish-doainant , or bilingual "Englisb-dooinant. 
There are no'/iidis, no Spanish. reading teachers, and no bilingual pro- 
gran, non^teaching support personnel at the Model B* schools. However, 
Model B teachers nay go to Potter-Thoaas School to consult with support 
personnel as needed. 

Tine involved — Each day Model B teachers^are assisted for 45 
ninutes by an English as a Second Language teacher and for another 
45 ninutes by a Spanish as a Se<jond Language teScher.' These two periods 
^re used to provide instruction and study in English as a Second Langtiage 
and' Spanish as a Second Language, respectively. During the rest of the 
day. Model B teachers try to devote equal ^ tine to' instruction in both 
languages in social stytdies, nathenatics, sciences, and supplementary 
activities. 

ARRIBA 

Grade levels, yedrs of operation, size — Arriba -began in ^1969 
and was designed' to serve newly inaigrated students who speak vii^^n^fQy 
no Ehglish, as well as Eng^^^sh speakers interested in Puerto Rican 
.history and culture. During the 1973-74 school ^ear, Arriba served 
1,050 students in grades 4' through 12 at three elementary schools, 
two junior high schools,^ and four senior high schools. School counselors 
parents, and teachers recooaended these students for place&ent in the 
'Arriba program. 
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Staff tag — At the elcaentary level, one bilingual teacher is respon- 
sible for froa 20 to 30 atudenta. If the bilingual teacher is English 
doainant, ahe aay provi<le 1-1/2 hours of English as a Second Unguage instruc 
tlon for her students. If the teacher, is Spanish-denlnant, she nay teaa w<th 
an Engliah-docinant teacher for English as a Second Language instruction. 
The Arriba teachers <Io not receive ^ny further instructional support or 
asaiarance* ^ , . * * ' 

Tiae involve — At the elcaentary level, 15 to 20 Arriba students - 
are placed in ungraded classrooas with two bilingual teachets.,^^Xach 
day, the English-doalnant ceaber of the bilingual teaching teaa 

'^provides 1-1/2 hours of English iLS .a"Sec6nd Language ins.tructloa ior, 

these students. All other classrooo activities are in Spanish. 

In the junior high and high school Arriba prograas, students also 
receive 1-1/2 hours of English Instruction a day. Students also nay 
chooae to study social studies, Spanislv language arts, cosaercial 
subjects, aathcaatic8»' science, and susic classes conducted entirely 
in Spanish. Couswe content for these classes is planned to parallel • 
the regular school prograa, and Arriba students join non-Sp^ish- 
doainant students In physical education; ausic, art, health, and 
black histoVy classes. • - 

* * • . 

, Currlculua and M&terialS ' ' ./ , 

A full-tlae currlculua coordinator , 'ftsiir full-tlae teachers on 
special assignaent, and 22 teachers' working part tine as currlculua 
developers consult with adninlstrators .and teachers to deteraine what 
naterials should be adopted, adapted, or developed to-serve Model A, 
Model B, and Arriba students. ' ' , ^ • 

Cosaercial naterials adopted by the -prograa include Laidlaw and 
Lippincott basal read^.ng "series, Schneider's science series, Finocchiaro 
and Lado*» Engliah as a Second Language naterials, Addiaon-Wesley *a 
HathegaticB tn Spanish , and nuaeroua audiovisual aids. 

Adopted natsrlalssinclude translation into Spanish of locally 
developed social studies units, the development of an ^udio-lingual 
approach to beginning oral Spanish, an<J phonics instruction to accom- 
pany the Laidlaw. reading jeries. ' ^ 

Materials developed ^to fit local program specifications inclu3e' 
i series of short stories written to reinforce the learning of con- 
sonants, '^nd flash car.ds to^teach vocabulary and usage. » 

In general, language learning nateVials used by the Let's Be ^ 
\ Anigos prograa stress the use of the audio-lingual method (frequent 
' repetition, pattern response drills, neaorization of dialogs, and 
translation only when necessary). Materials also tend to be action- 
oriented and include suggestions for songs, dancesr and art projects. 
Audiovisual aateriaXs including film strips, flash cards, and cassette 
players- are also recccsaended the curriculum development staff. 
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Inservlce-FreBervlce Training 

Preservlce and Inservlce training *f or the Let'« Be Anlgos 
progrta Is conducted by prograia supervisors, profesaors froo Templfe 
University, outside consultants, the program director, *Tid program' 
teachers. Twice a year, two series of ?t«ff development meetlngi are 
offered to all bilingual program staff members. To encourage attendance, 
participants are paitJ. "A series might l)e devoted to such area^^as^English 
as a Second Language Spanish' as a Second Language, science, mathematics, 
or classroom management. OccTasional meetings for all district personnel 
are scheduled to acquaint school system staff with the bilingual program 
and with the history and s»lture of the Puerto Rican. A t^lcal .meeting 
might feature a speech by the President of the University of Puerto'Rioo, ^ 
videotapes of bilingual classroom sessions, or a locally developed film 
on Puferto, Rican history and culture. * ' I ^ 

Each teacher in the bilingual program is observed on an average of 
once a week by her bilingual program supervisor, four t^es a year by her 
principal, and intermit^tantly by 'research staff membera and school-based 
reading and mathematics siJecialists. After each of these vtsits the 
observer and tlie teacher meet to discuss methods, materials, and classroom 
management techniques. , ^ ^ 

Native Spanish speakers who have completed 60' semester hours of > 
coUege, who, have taught in other countries, or who have participated 
actively in conUaunity 'affairs may apply to attend eight-week' sumer 
institutes co-sponsored by the bilingual program and Temple University. 
The Pennsylvania State Department of Education issues irftern teaching 
certificates to participants who successfully complete the institute. 
These participants ^hen atteml. summer and evening clashes to fulfill 
requirements for a regular teaching creden&lal* 

' Parent-Community Involvement 

Each week the Coordinator of Special Projects and a teacher on 
special assignment meet with a different group^ of from four to six 
' parents. Meetings ^re held at one of the parent's homes. At these 
meetings, the Coordinator of Special Projects or the teacher on special^ 
assignment explains the bilingual program's goals, answers 4>arents-', ques- 
"tions regarding bilingual education, .and encourages parents to attend ^: 
monthly Advisory Committee meetings. Parents are allowed to bring their . 
children to Advisory Committee-meetings, and local eBter^ainoent follows 
most of these meetings. 

The Coordinator of Special Projects and the teacher on special 
assignment also serve as liaisons and interpreters when parents wish 
to speak^'with principals, iq, one inatanc^, the result af such a-^eet- 
• ing was' to obtain a bilingual parent, to lead th^ home-school association. 
To keep the community informed about' the program, the "Cdordinator of 
Special Projccti produces a weekly radio ^how entitled ^/^ef-^ Be Amigos." 
The program format features student talent, talks witK Barents and local 
citizens, and Is geared to community Interests. Although it ioe's 
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not focus on program publicity, t>ie exposure given to the .Coordlnatoif . 
enhances her-ef-fscttveness^as. a p^rogram leader. 

/i " ~" "7 —7 - . • — --V- - 

Program Monitoring ... ' 

Each suBiaer the head of the research dlvlalon discuaaes results 
of the previous year's evaluation with bilingual proj.ect staff mambers .^^^gpts^ 
He then helpa therataff members wflte ^revised, measurable program 
objectives for the coming year. The head tht research, department 
and his two aaalatanta . then supervise program Implementation on a 
-rotating basis. They visit the bilingual program schools on an 
average oi^twict a week' to supervise testing, to test individual 
students, and to see that -strategies in use match ^strategies stated 
in* the proposal. ♦ ^ ^ , ^ 

« ' ? 

On an^ average of once a week, four bilingual program supervisors visit 
.program teachers for whom they aife respjjnsible. During these visits, 
the' supervisors check teachers' lea^son "plans » teachers in action, 
and discuss mutual concerns, ^ 

Reading specialists, laathematics specialists, and on-site coordinators 
visit bilingual teachers* classrooms as needed. In addition, the school 
district requires that each school principal observe all of his teachers ♦ 
in action at least twice a year. 



Cost 



Budgets and enrollments for the first five years of program operation 



were as 
Year . 


follows: 
Enrollment 


Tltl« VII 
Budget 


Total Budget (inc. 
other federal funds 
and local funds) 


TlMe VII 
- Per-Pupll 
Costs 


Total 
Per-Pupll 
Costs 


YeVr 1 
1969r70 


1,000' 


$197,927- 


$313,927 


-$197.93 


$313.93 


Year 2 

'1970-71 


* 

1,346 


$438,092 


$940,227^ 
' 51, 065, 348 


$325.48 


$698.53 


Year 3 
1971-72 


1,813-" 


$518,413 


$285.94 


$567.61 


Year 4 
1972-73 


2,705 


$S27,013* • 


$1,269,276 


$194.83 


' $469.23 


Year 5 

1973-74 


2, .830 


$695,066*""^ 


$1,645,066 


i 

$245.61 ' 


'$581.30 



* Includes the Regional Curriculum Adaptation Center, budget . 
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As can U' seen- Itt the flver^ycar table ,-*total monies allocated to the program 
have increased, st^^adlly^' There are several reasons for the steady increase: 

• The 'increased per-pupil cdst in Year Z, reflects- the ^expansion . 
of^the Arriija component into two more high schools and the' 
implementation ^of Model . ^- - - ^ 

• Teacher salaries rose in Year 3 due' to a retroactive pay 
^ increase totalling $40,000 for project teachers,. 

• implementation and maintenance of a Regional Curriculum 
'* Adaptati^^n Center increased project iosts beginning in 

Year 4. ■ ' \ - . 

• A strike ^^ly .in the program's fifth year of operation 
' brought teachers another pay increase.^ 

• As a result of the strike also, class size was limited to 

a maximum enrollment of 35 students per class; it therefore 

required more\ teachers to cover the additional classes 

which were created to maintain the maximum enrollment celling. 

\ 

Ttie program's rsliance gn federal funds, other than Title VII increased 
ffom $33,000 in Year 1 to $245^000 in Year 5. Local support for the program 
increseed from $83,000 in Xesr 1 to $705,000 in Year 5— nearly half of the 
total program cost. Beginning in Year 4, the local school district paid 
the salaries of all Model A teachters and*the majority of Model B teachers. 
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EVIDEKCK 0? ErTECTIVQ^ESS 
* 

. , Introduction . £va}uators in the Philadelphia Bilingual Prozac espha*- 
slze^'foraativje. evaluation.' The -prograa is evolving oad pro^raa changes hav 
been;and are being nade on the bdsis of the continuous feedback^ provided by 
the evaluation. For*exasple, because of t)ie 1973-74 evaluation )'^cr eased - 
eaphasli is now being placed on*£ngljiSh gains by Spanlsh-doainant studeo^^. 

. ' /. . . . 

,The p.ro^r*aJ^ worthy of consideration as aa i^xeapliry 6nt for two v^iqr 

reasons. First, evliluation results "have Seen favorable to tha prograa iihan ^ 
assessaent h'as^baen based on ccoparisons between prograa and nonprograa 
students. Host prograa objectives have 'been oet or surpassed. 

- ' • t • • • *' 

^ecdnd, c^ie progran evaluation design is. refrlicable and' has utility for 

' improving proves, evaluation ijj. other bilingual education progran^, It should 
also be noted that Philadelphia is one of the few bilingual edilcation proj^^cts 
reviewed by AIH that has classified all rested students on the basis of th'eir 
language doaina^ce, and' has reported results in first and second language per- 
formance T Such a^baaic concept in evaluating bilingual education proiectsf' 
has been frequently overlooked. ^ 
' * • ' ' ' ^ A ' 

> ^valuator . ^ The evaluation was conducted by Dr. Robert Offenberg and his 
staff.* Dr. Offenbjcrg a research associate with the Philadelphia School 
District and is assigned as a full-tloe evaluator for the bilingtial prc/ject. 

Evaluation year . 1973-7A. * ^ , ^ 

Aspects addressed . The evidence of effectiveness sunavy addresses the 
following aspects of the prograa evaluation: - ' , . 

• Language developnent — Spanish Kother Tongue ' 
^ • " ' English Mother Tongue 

. • ^ English a$ a Second Language - 

Spanish as a Second Language 

• $elf-concept ' * • * 

• Drop-out rate - ^ 

• Principal^' •artd parents* perceptions " 

•< Language Developnent ' ' 

Spanish ^jother Tongue - ^ • • , ^ 

■Sanpliag .' Spanish-doninant students intl^odels A and eletsentary 

and Junior high school students of the Arriba component in grades A through 8 
were tested on Spanish instruments. Scores for all students who. were present 
at the tisie of the testing. and who cocapleted all, subtests are ^reported. Sope 
subtests were administered on different days, and therefore soae students cuq^ 
hsve been present only during part of the testing. Because of the nany project 
adoinistered measures as well as district testing, make-ups were^ not arranged 
for ' students absent during the testing periqd. , 
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Cosparlaon . Treatacnt scudtacs art cocp«r«d to- « pre-progr«i baa rlint , 
Thi« b«8cUnc was e«c«bli«hcd In 1968 by cc«tlng aU jSp«al«b-sp»«king st«d«M 
in the Philadelphia schools- i^cv^ev^teen as needing specicl acttntlon due co 
limiced Engllsh-speakinf •btj^fy. This covparison ^is_ sees as rea s o n able be- 
cause Che saae cype of crice^ has been osKd for decersinin^ parcicip&clpn 
in the prograa and in the developing corriculuA, Grades 2,.^tJhrough 9 vcre 
adainiscered the saae ' Spa^iish tests used presently by the procraa* The 
baseline and treacaent itroups are cospared- to rural Puarte Rican noras» since 
the majority of students entering the program are reported as co«i«c fro« 
rural*Puerto Rican areas. 

> * ' .' * • ' ^ 

Keasures and data collection - Students in^the baseHne group and treAt- 
aent groups,* tested in Hay, ^968 and Kay, 197A respectlTely, have been tested 
in Spanish on the following aeasures: , 



•1 



Grades 1-3 Test de Destreza^ Basica^ en* Lecture 

"Prue'ba de Lecture 
'Int^r-A^rican Series, Level IV 



Results * Table 1 sbcvs the results on the Spanish langxiage tests admini- 
stered to Model A, Model 3(f and Arriba students, as veil as liaseline group 
results vh«n these were avkiM^. The following statistical testTs vere made 
on the data suacarised in Table 1. ' ' 



* Grades 2 and 3 : On the Test de Pestrezas Basicas en Lectuta tfisAxxt- • 
stered to pupils .in Model A and Model B in Grades 2 and 3, an analysis of var-, 
glance was calculated that indicated a significant grade effect (Grade 3 higher, . 
*p - -< .001), a signifigant prograa effect (Model A->igher, p « < .601) and no 
significant interaction. Two planned orthogonal coqparisons were cade: (1) * 
pupils in Models A and. B in Grade. 2 snowed -a slgnif icsntly higher (p - ^ .001) 
oean than did the baseline group; (2) pupils in Models \^ And B 1^ Grade ' . 3 flhnvf^ 
a significantly higher (p • < .001) aean than did' the bS^cline group. 

Grades 4 and 5 ; Scores on the Prueba de' Lectura Nf or Model A pupils 
In Grades^ ajSd 5 were cocpared with ^corei for baseline groups in .-an analysis 
of variance. There were significant prograa effects (Models A and B pupil seans 
higher .than baseline, p - < .l501>, signifi^t grade effects (Grade. 5. higher » 
p .002^ and no significiant interaction. / » 

Grades .J>>and 8 ; • Scores on the^J^Mjiarfi vetiion of the Inter-Aaerican 
Reading Test for' Arriba students in. Grades 7 and 8 were conpared, with pre-progran 
baseline pupils in an_^nalysis of variance sutaaary. Progratf effects were sig- 
nificant (p one tailed), indicating Arriba pupils out-perfomed^^the pre- 
prograa baseline group. Grade effeqce were also significant (p -<f01) favor 
of the Grade B students, and no sigdif leapt interaction was reye^J.ed. 



English'' Mother Tongue _ . ^ , . c 

Sanpling . Engl'lsh-doainan^l'students in Model A in Grades K throufeh 5 who\ 
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vere present during che ciae of ^he cesclag and'vfoo cospleced all subcescs 
were ceaced. Because of che stay projecc-adainiscered measures » as veil as 
discricc cesclng, cake-nzps were arranged for studepts abse^ during the 
cescing period. Model A is the only one of the three oodels ^steaatically 
•crviag £Qglish'>doalrift7it students* ^ 

' Coeparison . Tres t oent students art. compared Co a pre-progras baseline.. . ' 
Since tbt seasures used are che seise as are used in the district testing* the 
baseline group is cocprised of students at Potter-Thoeas School vho vere at 
the given gradt level the jear before the prograa vas^cyded upvard to that 
grade* Grade equivalents are used for cocparlson purposes. . ^ 

t&iltiple analysis of variance vas used to cocpare Hodel A pupils* scores « 
on each test battery with pre-prograa -baseline students. For each test battery, 
the* vc^ctor of aean 'subtest scores for Model A pupils is cospared with the vec- 
tor of aean subtest scores for the ^re-prograa baseline* group. Also reported 
are univariate ?>tests of group aean differences for each subtest in each 
battery. 



Measures and data collection . Baseline groups and siibsequent treataent 
groups have been tested in the Spring on the foUoving aeasures: 

Gr^e 2 : Stanford.>Achieveaent Test^ Prlaary Battery II 



Grade 3-5: Iowa Test of Basic Slcill 



\ 



Results , table 2 presents seans and stSAuard deviations for Hodel A and 
baseline pupils in Grades 2/3, A, and 5 on th\ Engll^ language test adoinl* 

TABLE 2 \ \ 
1973-7A Language Developaent: ^ Engli8t\Hother Vongue 
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SD 
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88 
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0.7 
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.2.6 
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0.9^ 
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2.8 
' 2.3^ 


0.7 
0.5 


^ Grade 




Mode-I A 
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56 
7A 


3.5 
3. A 
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3.2 
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Grade 
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Model A 
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33 ' 
56 
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1.1 
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1.0 
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Grade 2 ; The KASOVA overall te«t revealed signtf Icaat xaetn «TJbte«c 
differences betveen Itodel A aod baseline pupils on the Stanford Achievoent 
Test Priaary II (p • < .001). Also, univariate F-test» for eAch subtest showed 
scan differences favoring the Kodtl A pupils* for all subtests (p - .001). 

Grades 3. 4 « and 5 : The KANOVA suaeaary indicated significant grade 
nora effects (p • < .001)/ significant mean vector differ^es favoring the 
Model A over the baseline groups (p - < .001), and «l»o a significant interaction 
of Grade X Prograa (p -<.001). In every grade. Model A pupils showed higher 
gi^ade equivalents than did baseline pupils on .each subtest.' Univariate F-tests 
also indicated significant aean differences* favoring prograa pupila- on each sub- 
test. (It is- noted that this test of progi;^ effectiveness is the coc^arisoQ 
of scans for allJiodel A pupils in Grades 3, 4, and 5 versus the oeans of l^ase- 
line pupils in Grades 3, -4, and 5.) 

. ' . 

English as the Second Language . 

Saapling . Spanish-doainant students in ^els A and B and the Arriba 
cosponent in Grades 3,- 4, and 5 who vere present on the day of the testing 
were included in ,the sasple. . In Model A, students In the regular English as 
a Second Language prograa were Included. (Students new to the prograa for 
whoa the ci::phaliis was oral second language developaent vere not tested.) Vo, 
Model 3 end Arriba, all. Spanish-doainant students who had "any experience" in 
reading English were included. * , 

Cooparison . Spanish-doainant treataent students are coapared to the norms^ 
for English-speaking students. The expectancy was that Model A and B thir-d 
grade students would perfora two years below grade level— at a grade equiva- 
lent of l.S— and that fourth and fifth grade Model A- and B students wotad ^ 
grow at a rate of one year grade equivalent for each grade r«vel (2.8 and 
3.8 respectively), Ito clear-cut expectancy was set for students in the 
Arriba coapoatnt since tncy were recent inral^aats who had not had continuous 
education In Philadelphia schoolsi ' ' ^ 

Measures, four subtests of the Stanford Achieveaent Test, Prlnary Battery ^ 
II were adainistered: Word Meaning, Paragraph Meaning, Spelling and Word Study , 
'Skills. 3 

Results . Table 3 presents the sumaries by grade level on the English as 
a Second Language conponent. Results were nixed when co25)ared with expected 
perforaance. In third grade. Model A achieved the criterion level and Model B 
did not. In fourth grade. Model B achieved, the criterion leyfl and Mgdel A 
did not. In fifth grade, Model A did not reach the criterion ievel «nd Model B 
had not 'S^et been cycled up to the fifth grade level, (It should be pointed out 
thaf the fifth grade* group is the "pioneer group" in Model A, with 1973-74 being 
the first ycsr thst the project -ciperated at that l«vel,) 
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Spanish as a Second Langaag'e (SSL) 

Saspllng * All Engllsh-doslnaot students in Hodel -A participcting la regular 
SSL instruction in fourth and fifth grades were rested. Kev students to the 
progras for vhoa the esphasis vss oral second language developoent vere 
excluded. All students present on the dsys of the ^est ad^nistrstion vere 
ry sained ♦ . ^ 

Kessures . Test de Destrezas Sasicss en Ltctura . designed £or*flrst through 
third grsde native Spanish-speaking students* 

Cospsrison . SSL students sre cospsred to rural Puerto Rlcan, thij^grade^ 
end-of-the-year norxts. 



rto Kican, toij^grade^ 
ear, third-^grsda norss, 



Results . Cocpsred to rural Puerto Rican, ead-of-year, 
English-doainant students at both fourth and fifth grade l^els vere perforalng 
nesr or sbove the noms« The Letter and Vor^ Recognition subtest reqt^res 
xaore knowledge of phonics than of Spanish langxiage, vhile the Vord Meaning 
and Parsgrsph Meaning subtests do require knowledge of the Spanish langjiage. 

Tsble 4 presents the^ Deans, standard deviations, and percentile ranks 
on ^he Tfcat de Destrezas Kasicas en Lcctura for pupila in grades 4 and 5. 

, TABLE 4 

"Perforriant^ of English-Ooalnant Fourth- and Fifth- Grade Pupils ^ 
On Spanish Reading Test, Test 0e Destrezas Bosicas En Lectura 





Letter anji Vord 


»Vdrd 


Paragraph 




Itea 


^Recognition' 


Meaning 


Meaning 


Cocposic^ 




N X SD PR* 


X SD PR 


* 

N X SD PR 


K X SD , PR « 



^Grade 4- . 78. 60 5 80 • 76 14 6 40 83 13 ."S- 50 74 87 13 60 
Grsds 5 32 62 5 90 32 16 '7 5(5 31 16 5 ' 66 30 95 15 70 

* Psrcentile rsnk Is bssed on rural norms of Puerto Rico, third grade and- • 
of— thfe-)«sc. , 
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Self -Concept 

S S^dla^* 166 $p«nl8h-doclnant studeata* were r*ndoaIy selected from 
the Kodel A and Arrib* coepoaencs 4p -fourth through twelfth graile*. 

COttpArlaan . BtUngual treatseot studeata are coepared 'to 100 randomly 
aelectcd students participating in the dlatrict English aa a Second Language 
progsaa in achoola where 'there is^o regular bilingual prograa. For analysis 
purpoaea, the studeats were divided into three groups: eleacntary (grades 4-6-), 
junior high school (grades. 7-9), and high-school (grades' 10-12). 

Measure 'and data coU'ection . .The inatruaenc used was a 'translation of the . 
Cobperaaith ^If-Esteea (Snventory v The inatriiaent was translate by a proiect ^ . ^ , 

staff aeaber and field tested and revised the-s^sier prior to,^ use. If was 
orally administered to ssall groups by a bilingual nesber of the project* evaluation 
staff. ' 

Results. The antfl>-sis 'shows that there ^ were - statistically significant ^ * 

differences between the prograns and*between the grade levels and a significant 
interaction between these two vari abler.; At the eleacntary and junior liigh 

school levels, pupil self-esteea is siaiiar. However, there is a strbng / . 

difference "between tl)e two groups at the senior high school level, with the ' ' 

bilingual pxojept participants clearly deapnstrating a 'higher level of splf- 
«Mtata. Resu»lts are graphically illustrated in Figure 1. ^ 




< , Prop'^Oat Rate J. 

Sampling. Air bilingual prograsi students In thfr Arriba coi^)e»aent vho were 
In tht tmath irrade In D«caaber» 1971 tnd who vere claatlfled aa ^panlih apaaklng 
ln,tht dty-vSda pupil dlrec^or^ ware' Included I9 the aai^la (H-»i04) 

Coaparlgoa ^ The bilingual trtattacnt aaapie vas co«p*red to all othtr atudents 
In the treat»ent <5^ool», and clty-wlde, vho wart Itated in tha cUy-vlda pupil 
directory aa Spanlah apeaklng but who wwra not partlclpattiig in the bilingual 
project. THe cosparlaon group conaiated of 264 atudenca noi; A^./jfl ^ba, hut in 
'the aaae achooZa, and 451 atudenta clty-vtda. ftie coaparl«on^l!^«latad of the 
.percentage of a'cudan'ta enrolled in December, 1971, who were gradnated in June, 
1974. Jhla longitudinal, approach vaa takctv to control for aumer drop-outa between 
grade levela. ^Ithin-acholastlc year analyses done in previous years have also 
shown conslstentily favorable results.) -{Results are shown in Table 5. 

TABLE 5 



Graduation Rate of Spanlsh-Doninant Students Who Were in Gr^de 10 in Decenber 1971 



Group* 


Graduated 


Kot 


Graduated 




Total 




\ H 


X 




X 


N 


X ' 




In Arriba 


\ 

• 30 . 


Z9X 


* 74 


71t • 


104- 


1002 




Hot in Arriba 


(aaae schoola) 52 


20% 


212 


8or 


264 


lOOZ 




Hot in Arriba 


(cltx-wlde) 115* 


26r^ 


336 


75Z 


451 


lOOZ— 





Results . BlUttgual project students were taore -likely t6 graduate than • 
other Spanish-dominant students in tKe saae, schools. The dtfi^^^^e was statis- 
tically significant "at the .05 level of probability. When coiq)ared-to city-wide 
Span^ih-domlnant students, the. difference was not statilK;;ically significant, bur 
favored the bifingual project.* (See Table 5.) ' ' '" 
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• ^Opinion Surveys , 

Suapiry . Opinion surveys of prlnclp^s of project schools and parents of 
1>articipatlng students were conducted In M«y» 1974. Principals in the eleven 
participating scho61s vere asked for chelr general, overall iepresslon of the 
project and for specific inforcatlon about pupil and parent reactions to it. 
Parents vere asked for their perception of the j>rograp and about their participation 
in and contact with the project". 

Saagle. Principals : All eleven principals of participating schools 
-responded. 

\ Parents ; A total of 677 parents responded, or 34X of the 

^ ' parents of participating students. 

Measures , Principals vere nailed questionnaires in Hay, 197A. The parents ' 
questionnaire vas in English and Spanish. * It vas sent to the parents vith all ^ 
the students vho vere present on the day the questionnaire vas distributed. Since 
the questionnaire vas anonyaous, there vas no vay of follovlng up on chose vhlch 
vere not returned. * 

Results for the Principal Questionnaire . The questionnaire consisted of 
seven questions. The flrsc tvo questions vere general, dealing vith overall 
satisfaction vith the project and opinions about expansion. Questions 3 
through 6 dealt vith teachet* supervision, the susaer Institute, and increased 
understanding of the project by parents, students, and staff. Question 7 
asked for additional areas of concern that principals wanted to bring to the > ^ 
attention of project oanagesaent. 4t ' 

Dealing vith tha tvo general questions about the project,* the responses , 
vere as follovs: 

a. pverall level' of satlsf-actlon vith the bilingual project: 3 
vere satisfied, 6 were soDevhat satisfied; none vere soioevhat 
dissatisfied or very dissatisfied*. ' \ * 

b. Opinions aboutSlpandl^g theev'i^ograo vlth^n their schools: 4 
expressed desd^^that the program be expai^ded to reach more students; 
7 felt that the program should remain the same size (5 responded, 

."already served children vho need it," 3 responded, '^'l^xitations 
in terms of spade and teacher vacancies t'o allov employment of 
bilingual teacheiJfl"; no one felt' the program should be reduced, 
or eliminated. * ' . - 
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g.^uH^^-^or th^ l>^r«.tL Ouestfonnalre ^ athough caution »i«t be exercised 
In vleving these dat* because of self-selecUon of the respondents (3AZ 
response rate), the results Indicate a high level of support foe the progr 
The questionnaire is divided into three sections:' (a) hooe language use, 
(b) perception of school prog^an^ and (c) parent participation. 

^OnThrce questions dealing with 4>crception of the school, program, the 
parents responded in the following manner: 

^ a- 93X of thf parents felt that thtir children lilced learning • 
Spanish and English in school. 

■ b. 94X of the parents lilced their children to be learning Spanish 

and English In school. . » 

* 

c. 93X of the parents wanted their children to study prolangoages 
the following year. 
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PROGRAM AREA : 

Blltngu«l Education (Sptnish/En^ll'sh) 
PROJECT TITLE ; - 

Alice Independenc School District Bilingual Education Prpgran 
LOCATION ; 

Alice I Texas 

SOURCE AKD-LEVEL OF FUNDING ; ' ' 

The following figures are for the fourth year of operation. 
Title VII Funds $100,625 

Other Federal Funds 75,820 , - ' 

State and Local Funds 201>167 

Total' ' $577,612- 

PROGRAM START DATE ; 

1970 ' . 

t 

BRIEF DESCRIPTION OF PROJECT : * . " { ' / 

Goals aftd oblfectives . It is the goal of the project to teach children wlt"h 
United English-speaTcing ability to speak English while also developing profit- 
ciency in SpcTnish. It is felt that developing both languages vill facilitate 
Achievement in school. It is also the^intent to develop and maintain the * 
children'^ pride in themselves and both cultures and to involve parents from 
non- English-speaking environments in school-related activities. 

Context . '.Alice is a small, semi- industrial town of about 20,000 people. -It 
is ^O'm'iles west of Corpus Christi and 120 miles north of the U.S. /Mexican bor- 
der town'' of .Hidalgo, Texas. ..Much of the enpldyment is in services related to 
.Oil«lnin|." the population of Alice is very stable, with' the exception of 
migrant wDrkers, many of whom' return to Alice each year after their flel4 work 
it finished. There is some mobility within the town, b^t attrition from the 
program has notr^.been a probrca.. 

the total enrollment in the Alice Independent School Diatrict i? 6,^68. 
About 69X are Mexican-American, and 31% are Anglo-American. It is estimated 
that 66% of the 'total enroXlnent has limited facility in the English language. 
The proportion of Mexican-Ame*ricans in the four .elementary schools' of faring 
bilingual education in' the 1973^74 year wa^ about ^6X. 



Major features of ,the progt;aci-arl^the following: 



• the use of performance objectives for all subjects taught 
bilingually»\ ' ' " ^ . • - ^ 
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• cetcher ptrciclptclon Ir^ deceraitilng certain eleaents o£ 
the curricula* - 

• ronpri»h*»nslve rerord-kccping, . ■ • 

• flexibility.* of instriictlcJnal approach, . - , 

• Intensive preaervlce and Inservlce itralnlng, and 

t active parent and cocaunlty involvement* , ^ ' • ' 

These features vlll be discussed in approprl4te sections belowl 

Program description < 

Years of operation, size, grade levels — In 1973-74, there were" 52& 
child ren in grades' K->4 in fout schools. This year the program has expanded 
horizontally Into all nine elementary schools, two of which are paroehlal. 
It has expanded vertically Into the 5th grade. - ^ 

Staff — Staff members who- were full time on the bilingual project in 
1973-74.<were the director, an evaluator/materlals specialist, 19 teachers, 
6 teacher aides, and a secretary. All central staff Are bilingual. There were, 
also 7 mother aides who had children In the program and^vho .worked part time. 
Staff from other programs administered by the district (i.e.'. Title I, Migrant 
Education, ifcireer Education, Comaupity Involvement) freely share their resources 
and experience wi€h the 'Title VII staff. 

Curricula and time Involved — Children generally enter the program in 
kindergarten. They are preregistered by Their parents In the spring before 
they enter kindergarten andare accepted on a "first-come, first-serve" bssls. 
A language proficiency test, provided by the Texas Education Agency, is adminis- 
tered in English first and then in Spanish to determine the child's proficiency 
in "feach language/ ^ '* ^ . ' 

• The •instructional component for all grade levels denters arojuild performance 
objectives in the following content areas: languag;e arts tincluding listening, 
speaking, reading readiness* and reading skills), taathematics,* <:uXture a&d 
heritage* and sciencer T^e objectives^ which .ware developed by program staff, 
•ra written in beKavioral terms. Mas^cty tests'for each objective have also been 
davalopad. For each objective «t everyj grade level* 'there. is a designation as to 
^hich language (s) the objective should /be mattered in and what percentage of 
students are expected to master it. 'T(1(b objectives are revised avary suoMr by 
teachers. During the year, if 'teacharff discover that an objactivf it too taty, 
too difficult, or somehov InapprDpritt^, they consult ylth the director'' and 
avaluator and either drop the objectiv^' or rewrite it. ' ' _ 

Teachers submit lesson plans to the program director every**two weeks that 
indicate which objective^ are'being taught, fn which language they are* being 
taught,, and types of activities planned. The director reviews these lesson 
plans and returns them to the teacher^. 

. i' % * ' 

Detailed records are maintained on each child in the program indicating how 
many objectives he has mastered and his score on each mastery test. In addition 
to keeping a class r6st;er, teachers maintain a separate re^rd for each child. 



which beccpes part of hl5 pernaiicnt til^. This systea cuabl 
at the b«glnning of^ch^eAr cxaccly at^vh«T level each chi 
Racord* are ipot-checlced throughout the j^ar 6y the director. 

Instruction begins in kindergarten In Spsnish in teading 
Isnguage arts^ and macheaati^s. Other activities such aa group 
nornlng, story t:ine, aysic, art, and afternoon review are also conducted In 
Spanish* Lasiguage arts in EngHsh concentrates on developing oral language ^ 
skills and in highlighting the slsilarities and differences in the aounds and : 
alphabets. of both languages. Th^ concepts taught in Spanish are reinforced in 
Eng'lish, As kindergartners becooe proficient in Spanish and begin to. develop 
basic skills in English, aore tloe is devoted to instruction in English^ Xinder- 
garten children also view an early intervention progrss for preschool children 
carried on local cable, television. The iitogras consists of a 30<<iinute color 
broadcast each day produced at the Alice High School. Xc is bilingual and is 
de«J,gned to. upgrade ^he cognitive and*af fective sitills of preschool children in 
Alice. Concepts presented during the first 15-ainute segment are reinftftcad 
'with a story during the second half of the prograa. 

. ■ . ' ■ : ' ' 

Ueading instruction begins in grade 1 in Spanish by a "building block*' 
approach^ Children learn the sound of earh letter in^the Spanish alphabet ^ 
its nane, and how it looks, t^ey then put sounds together to sake syllables, 
tnd put syllables together to cake words. The> also begin manuscript vritlng 
in grade 1. English vocabulary and langxiage skills continue to be reinforced; 

late fall or early winter, the, reading readiness tes^ that sccpiipanies the 
district xeadl.ng series is ^adoinistered to snail groups of ^ first graders. Based 
on results of this test and teacher observation> reading instruction begins in 
English. For all ctiildren .this is by January df grade 1. As the child's 
perforaance in Spanish itsproves and his )}roficiency in English develops, equal tioe 
is spent instiructin^ in both Itngtiages in language arts, cath, and culture and 
hsritage. For^caost children, this is during grade 1. , 

In grades 2 throug\ 5", instruction continue? to be given in English agd in . ^ 
Spanish for equsl aaount8.^of tloe* Soae teachera teac^ on*''vcek in Spanish and 
the following in English; others alternate^ every other day* Reading in* Spani^ 
and in English is taught dally. The prograa provides teachers a great dial of > 
flexijjility, and they are free to' use their own jiid^ents about- when is the wpt 
time to move into completely billnguaJL instruction. ;n 

At all grade levels, program classrooms are heterogenous and self-contained,* 
and teachifig methods vary. Grouping is generally informal and is determined .^y 
language proficiency *arid/or achievemeat levels There is a» shifting emphasis to 
greater individualizing of instruction, although this approach is not imposed 
as protocol on teachers. Some teachers use interest centers as a means of, pro** 
vidlng a variety of learning experiences; others eatploy more traditional techniques » 

' ', Haterials^ -The following is a>^ summary of key materials used at all * 

grade levels. ^ ^ . * ^ -"^ * * ' 

* Reading r'eaJinesV skills ere 'taught in Spahlsh and Engiish^*uaing 
« Peabody Language Jevelopdent Kita, ROCK materials developed by 
\ the Region ^e^BiWcation Service ^Center in Edinburg, Texas, The j 
i Alphsbet- Book and Really Heading. Series published by Harper and 

Row, and Alpha Time with, the Huggables by "New Dlmei|dions in 
Education* • Teachers also have access to teaching machines, Ian- ' 
^ gusge kits, and othfer .audiovisual equipment. 



Reading Instruction in Spaaish is tifught using Ketodo , «w 

Oaoaatopeyico , Mi Libro Msgico froe the dontineatal Stries ^ 
. snd Prcpsrsndose Pars Leer by Houghton-Kif f llrf» The V*^idJI^> ^ 
Series, Por EI Kuado Del Cuento Y L* Aveotitfa, is the core ^ 
of the Spanish reading prograa in grades 2-5. * • V — 

Reading in English is taugbtjvith the district-adopted Harper 
Rov Reading Seriei\ 

Language art^ are taught in Spanish with Ejercicios de Lenguaje , 
a graacar book froa Mexico, and Our Language Today from the 

"X»rfcra5^3obTc"Coapany " " " = ' 

Supplementary isaterials for each reading series (vorkbooks, 

flashcards, charts, etc), staff -developed readers,* and cany > 

other library books in. Spanish and English enrich the reading 

and language arts progress. « 

Por oathcnatics instruction,' The Eleaentary School Hathesatics 

Series by Addison-Wesley is used throughout the district. ' * ^ 



; Science is taught bilingually in grades A and 5 using the 

McMillan text. Science for Toaorrov's World . The staff has * 
, . I written units in Spanish -ca aupplenent^ the text. -^^^^^ 

The staff has also developed a prograa to teach culture and 
heritage 6&ll6i ^I Jjerencia Cultural , which is adaptable to 
^ ' any krade leve^. *S^pe oL the tpji^juQlncltj^ed are the isaigra- 
* ' t ion. of i^exicans to "America, 'thi' ai&it*^-*^**®^ Mexican-Aaerican 

fam vorli^^rs, .and recognition of proainent Mexican^Anaricans 

' ' ^ in Alice. 

Facilities — The prograa operates in regular classrQoas* No aodifica- 
](^#«8^of facilities are required. 

. • ^ Preservice/inservice training— In August, air Title VII staff meobQrs 

attend ^ coaprehenslve preservice training session. During the suxsoer 1973* 
session^ which involved 80 hours* training, consultants in linguistics and South 
Te?^g^ Mexican llt^ature spoke to the/staff. Representative's "froa publishers 
als^Jneaonstrated^ neW materials. Teachers revised performance objectives* and 
wro9? aastcry tests for all ferade levels. They also^developed instructional 
naterials, which were evaluated by two consultant* from a nearby university. 
. / , ^ Staff also discussed proble^a encountered the previous year and sought solutions 
^ / J- r/^ii6 them. Teachers and aides rec^ve stipends for attending the workshops. 

i""' V»J^ During the school year,. 10 daya* are set aside for inservice training. For. 

C^v , V ^ Hl^^hers, programs center around educational television, taacher attitudes, 

(eachcr assessment, apd a review of performance objectives. T^ics for teacher 
• O **'-'and mother aides reflet the project's conDitment to involve asaistants ^totally 
in tho instructional process. Soae of their training incl<ides learning how to 
^ ^ listen to recitaAons, work with small groups, assist with drill exercises* and 
^**C work on special cultural projects with students. 
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Teacher pcrforsaace is videotaped twice a year using the ?l*o4er8 Ijater- 
actioa-^^ifsis, a syates designed to oeasure how =uch essph&aia the teacher 
placet on the contenC of what is *>eing preaented* on students' ideas, an<5 on 
the uae of authority. The director revtewa each teacher's tape with thjs. 
teacher. . ^ • 

t 

Other ataff development activitiea are vlaits to other hillngxial education 
prograna throughout the atate atkd attendance at conferencea in other parte of 
the country.' Taachara are also working toward receiving certification In bl-- 
lingual education, which is aoon to be aandatoiy In Texaa., Certification - . 
Involves taking a 3Q-hour claaa given by one of the regional offlcea, aCr^leaat 
one year teaching experience, and deaonatrated proficiency in Spanlah. * 

?arenc/c6gaunity Involvesent — To encourage parenta to bacoae active 
partlclpanta In their children' a education, priority haa been given to parent ' 
involvcnent in the project. The Title VII Adviaory Cosalttta aeata every aix 
weeka ot^ different school caspuses. Kesbership 4.a open to anyone interested In 
bilingual education. A constitution was -^nritten and approved during 1973-74 and 
has contributed to iaproving the Coslttee'a status as an organization. There 
are five officers. Throughout c^e year the Cosalttee has staged chlldren*a 
^prograas at their seetings, has set with oenbers of the Title I advisory group, 
and has helped teachers -present a Mother's Day prograa. The Co^oittee's 
approval of new proposals is also solicited. ^ 

N 

Parents volunteer to work in the school libraries and have prepared zany 
of the consumable materials used in claasrooss. Workshop» are held where they 
learn how to care for and repair library bookd> and how to perfors certain jobs ^ 
in the library. Encouraging parents to \fovk the library is seen as one way 
to atress the importance ot reading — an ecphaala that hopefully they will convey 
to the children. * 

Vorkahopa, generally scheduled by the Home Coordinator, are also held to 
teach parenta how to be effective learning ,ald» in achool and at hooe. 'Parenta 
are welcome in the claaarooma at any time and are willing volunteara when extra 
help ia needed, such aa on field trips or prepiring for a special program. 
Two achoola have ?arent Involvement Centara, equipped with a television, aofas, 
^zablaa, chaira," ahelvaa, and booka. Parant-teachar conferencea are held twice 
a year eo diacuaa how the child is proceeding through the program. 

Ditaeaination of information about the project to the public is another 
easential element of this component. Sotae of the resources tJtlllzed to keep 
the concaunity informed of t)i^bilingual events are newspaper articles, news- 
letters in Spanlah and Ejv^lsh, Christmaa cards, and announcements in the paper 
about every event in tfie project of public concern. 



Co»t»— Tot*l budgtt« «ad earoUacats for the 5ir«t four yeAr» of prosr«a 
operatloa vere 4t follovt: 



' Eorollafeat Total Budget Per Pupil Cost 

Year I 150 ' $101,150 $674, 

i ' ' . 

■ Year 2 270 , $285,150 ^ $i,056 

"* *^ * • 

Year 3* 335 $366,730 • $1,095 

. Year 4 * 528 $377,612 S715 

The per pupil coat la computed on the total coat of the prograa, including 
t teacher aalarlea, and la the aaouot expended for the tot«l edxicatlonal prograa 
for each child In the bilingual education prograa. The per pupil coat for the 
reguUr diatrlct prograa in Alice was S764 in 1972-73 (Year 3 of the bilingual 
prograa) and $785 in 1973--74 (Year 4) and inclodea all expcadituVea except 
bonded indebtedneaa and conatruction« 

a 

Kajor atart-up coata in Tear 1 vere for inatructionAl.aateriala and conaua- 
able auppliea C$12, 6A6), capital outlay for instructional equipment, furniture, 
and 'con-conauaabla inatructionar naterials ($9,63A), contracted services (_$2,000), 
and travel for 'ad«tnlatrat^.ve and teaching .atsff ($4,780). In geaeral> the 
aaounta spent for'aajor atart-up expense categories have dropped, except, of 
course, for staff aalaries and ins true tioaal taaterlals as th» prograa expands 
to include Bore classes and additional grade levels each ye^r. A significant 
rise in per-pupil costs occurred in Year 2 and Year 3, when this figure was about . 
$1,000 per child. This increase was 'incurred when teachers and paraprofessj^onals 
began to be paid for participating in the intcnaive preservlce training reginen. 
Theae atlpenda are paid to all participating teacherajand paraprofeasionala and 
aaounted to $3,690 in 1971, $5,700 in 1972 (Year 3)\ a^fJ $6,880 in 1973 (Year 4). 

Alao contributing to tW rift in per-pupil coat in Year 2 was the 
addition of paid taother aldea. In year 4, the per-pupil coat dropped back 
cloaer to the Year 1 figure, about $715 per child* The raaaon for the^drop» 
according to progrka ataff, vaa that the aaount apent for budget Items from 
previoua y<»ara did not incraaae proportionately with the expanding prolan 
enrollaent. For inatance, the aaount apent on oatsriala, equipment, and auppliea 
waa aharply reduced ao that the prograa could continue with local funda should 
Title VII nonies ceaae to be available. Prograa ataff found that caterlals could 
be catalogued' so that teachers 'could share what had already been, pur chaaed with'^ 
out detriaent'to the inatructional program. 
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EVIDSSCS 0? EFFECXrvSKESS: 

. Overvlev. During the pres«nc 1974-75 *cade=lc year, the bilingual educa- 
tion pro^as in the Alice Independent School District (AISD) is in operation in 
28 classes in nine schools. These are Saen^, Kayer, Salaeat, Garcia, Koonan, 
Rillcrest and Schallert in the public school systes, and St. Joseph's and 
St. Elizabeth's in the parochial systen. The first four schools vere those 
that had been continuously involved in the bilingual pVograa at A2SD for ^ve 
years since its inceptfbn in 1970» Since its start, the. prograa has grovn both 
horizontally (oore classes st a given grad« level) and vertically <expansion 
through higher grades). Table 1 shovs the* extent of the,^rogras*s grovth at 
AISD both in tems of totsl nuaber of classes and in the oc)6ber "of "^participants 

each yelr. ' , C r. 

■V •* 

TABLE 1 i • * 

Classes Within Grade Levels Vith Schools at' 
Alice Independent Scliool District 



Schools 


1970-71 


1971-72 


1972-73 


1973-74 


1974^75 


*Saenz 


1 


1.2 


K,1,2J3 


K-4 ' 




*Kayer 


1 


1.2 




X-4 


t-5 


*Salasar 


K 


K.l 


K,l,2 . 


K-3 


K-4 


*Gsrcis 


1.1 


K,l,2 


K.1,2,3 


K-4 


• K-5 


Koonan L 










1 


Hillcras.t 










1 


Schallert 










1 


4 

St. Joseph's 










1 


St. Elizabeth's- 




K 






1 


Total Clssses 


5 


9 


V 

15 


19 


28 


Total Students 


150 


270 


417 


528 


766 



Title I schools 

The evidence of effectiveness of the AISI^ bilingxul prograa is reported for the 
third and fourth year of the progran's operation; pre- snd posttest score.s and - 
gain scores on the Inter-Anerican Series Reading (Lecturs is the Spsnish version 
title) and Cenersl Ability (HSbilidad General is the Spanish varsion title) tests 
sx>e coapared for prograa snd control pupils enrolled in kindergsrten through the 
third grade in 1972-73 snd in kindergarten through the fourth grsde in 1973-74. 

> Evsl nation . The 1972-73 year's evalust^on of the AISD bilingual prograa 
vss nade by V. J. Kennedy, Educstional Prograa Auditor, University o| Houston; 
the ^973-74 yesr's evaluation vssr cade by Amoldo Salinas, Project Director, snd 
Clsuidina Hernandez, Project Evsluator. . i 
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YtkT aa^i ju-ade segples . Table 2 saovs the nuaber of progr^s and control 
group participants vitb coaplete pre- and posttest data en the Inter-Aacrican 

TASLS 2 

r 

3ilingual Prograa *od Control ?*jptla 
With Coaplete Pre- and Posttest Data 



. Grade 


1972- 


1973 


1973-1974 


3ilingual 


Control 


Bilingual 


Control 


K 


124 


32 


127 


33 


1 


104 


27 . 


106 




2 


94 


21 


95 


35 


3 


75 


22 


101 • 


29 


4. 






75 


20 


TOTALS 


397 


102 


504 


136 



3111ng\ial group* -Chlldrtn who tntar kindergarten in the Alice Inde- 
pendent School -District are tested vith the Oral Ungtiage EligibiUty Test for 
Kindergarten Pupils . This is an individually adaini^tered t€it to deteraiae 
oral English ability based upon the quality of verh^ responses to 20 questions. 
Children* perf oraing poorly on this test and vhb are registered for the bilingxial 
kindergarten see placed in the bilingual claas if space Is available* Bilingual 
classes never exceed 35 pupils so that participation in kindergarten is a func- 
tion of test score » parental approval and space. Kev children entering eleacntary 
grades who experience English language problems and vhose parents desire their 
xoVlaent in a bilingual prograa are placed in a bilingual class. (In 1972-73, 
e'«%hird-grade bilingual <?lAee w^s reduced in size because of a residential nove 
by several parents to a nevly constructed housing development. This class vas 
Increased by replaceaents vhose parents agreed to their participation when re- 
quested. These replaceaent children were average or above in ability since they ^ 
vould need to acquire Spanish language skills quickly; the group of children who 
left the progras at this tiae and their replaceaents were both Spanish doainant.) 

Control group — There is one control class at each grade level. The 
project director attespts to pick a regular class vhose pupils' Oral Lang\iage , 
Eligibility Test scores oatch the bilingual kindergarten class's Bost closely; 
this is then the control class for that year's entering kindergarten group. All 
control classes are looted in Title I schools. This control class is essentially 
a control cohort as it progresses through grades K/ 1^ 2, and so on» Kev childrem 
entering elsDentary grades placed vith the control cohort join it if both pre- 
and posttest data are available. So far, no control student has later becooe a 
bilingtul'prograa student, and no bilingual progr^a student a control' itudent later. 
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Cosparltoa— Pupilt in the billaguAl «nd control cUsset tre t&tlnit- 
ttr*d both, the Eagllth *nd Spanish verslont of the ReAdiag «nd Ceaeral Ability 
Tettt of the Int'Cr-AaericO Seriet tvice • ye«r. Pretetting it during the first 
tvo veek« in Septeaber; potttetting in sid-to-Ute April. Thl* retett tntervil 
it tpproxtnttely tcven oonths. 

For each Itngtitge version, different forat of the tettt are tdn l nl stered at 
pre- and pottt^sting. The control groups tt each testing are firtt tds^ttercd 
the Sptnith version, folloved the next day by the English version. Bilingual 
pupils first take the English version, cheo the Spanish. All testing takes place 
tn the aoming. Table 3 sxJasarizts the general pro<fedures foiloved by AISD for 
adainistering the Inter-Aaeripaa Series st each grade lyel. ^ 

• , TABLE 3 

Order of Test Adsinistration 



Pretest Posttest* 

Group I 2 1 2 

Bilingual ^ English 'Spsnish English Spanish 

Control Spanish English Spanish English 



sterec 



*In every case a different fona of the*, pretest. is sdalnl 
ss a posttest. 



Other tests— Beginning vith the 197A-75 acadcaic year, plans were 
isplcaented to also administer the Reading and Mathematics SRA Achleveaent - 
Series to all pupils in the elcaentary grades froa grade 2 upward in* AISD. 
Therefore the bilingual snd control gjroups .in grades 2-5 in 197A-75 have been 
pretested with the SRA Achieveaent Series fcnd are currently being posttested 
with different fonas; cosparstive gsin dats, as well as corpsrative data for 
the bilingusl and the entirs aainstrsan^group will be available in the susaer 
of 1975. I , ' , ' / 

hUstery tests sssocis^ed with the Harper and Row Resding Series sre sdain- 
istered, but no Cooperative bssel resding data suaaaries for the bilingual and 
regular students are prepsredt , « o 

Copparison procedures . Tor both the 1972-73 and 1973-74 school years, 
nean pretest, oean posttest, and aean grfin (posttest ninus pretest) on the 
Inrtr-Aserican Series Reading and General Ability Tests are reported for the 
control group and for the conbined bilingual classes at each grade level. 

Mean pretest* posttest and gain scores for the control and the bilingual 
prograa ^^roups enrolled in K through 3 in 1972-73 and X through A in .197^74 
are shown in Tsble 4. . 
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Ko significance cescs of sean gain difference have been re>orc«d alct^ugh 
^ the project evaluation reports refer to statistically significant differences 
In gains betveen project and control studenta. The rav ^ata suaoarles are 
uAavallabU. Bovever, It la possible to conaervatlvely estlaate the slgnifl- 
•cance of the nean gain differences between* the two groups. The following bases 
and assusptlons' are sade: • 

• J'osttest standard deviations are reported for claasr^os groups 
at^each grade level. Fooled estlsates for the bilingual grade 
groups are K - 7.0, 1 - 7.0, 2 - 12.0, 3 - 15.0, 4 - 20.0. ?re- 

^'^ xest standard deviations yill be assuaed to be* the saae; this is 

a conservative assusption since postteats nearly always show 
greater variability than do pretests. 

• There is no estimate fbr the correlation between pre- and post- 
tests. It is assuaed to be 0.00; this is a conservative assusp- 
tion since pre- and posttests in heterogenous groups ^re usually 
positively correlated. 

• The standard error of Dcan gaj^ differences is equal to 

751) * K(pjject) " ^ ^ ^^^^^\ 



/ 1 

/ K(cont 



* .25 /sD2pre + SD^post - 2r pre-post SO .pre SO post 
^ .25 Jl" SD post (pooled) 

- .35 S& post ' ^ 

A significant gain difference would be about twice its.esti- 
cated standard error or .70 SD post. Based on the above esti- 
siates, significant differences are K >> 5 points, 1 « 5 points, 
# 2 ■ 9 points, 3-11 point*, and 4 ■ 14 points. 

>g>lg6if leant results . 

^ 1973^74 groups — Every ncan difference favors the bilingual progran 

group. in the 1973-74 coaparlsbns (see Table 4). Conservative estixaates of the 
signiflicance of the mean gain difference between the project and control groups 
show *l8igniflcant" differences favoring the project 'groups in grades X, 3 and 4 
tji Entalish Reading, in grade 1 in English General Ability, in grades K, 1, ^Tand 
4 in Spanish Reading and In grades 1 and 4 In Spanish General A^lity.^ • 

In teres of educational significance, oean gain differences are ipproxioately 
equ^l |to one standard deviation (post test) in grades R and 1 on all four tests. 
Jn grade 2 tha oean differences were at least one-}ut^.of a standard deviation 
unit (posttest) except for Spanish General Ability. In grade 3, oean differences 
were at least three-quarters o^ *a standard deviation unit excep't for Spanish 
Generajl Ability. In grade 4, the oean differences were at leaat one- standard 
deviatjion unit except for English (General Ability. 
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1972-73 groups — All buc chi;ec Dc*n gain differences favor the bilingual 
' prograa (roup. The exctpcions are che cvo Versions of the General Abilicy Tescs 
in K, and che English tJtntral Ability Teat in 1. "Slgnlfic*Qt^ nean gain differ- 
ences favoring the prograa groups vere found in English Reading in grade 1, 
Spanish Jleadlng in 'grades 1, 2, and 3, 4nd in Spanish General Ability in grades 
•1 and 2. . 

^ 

As for the educational significance of the cocparlaons, the oean gain differ- 
encea vere approximately one standard deviation (posttast) or higher on^ all tests 
in ^rade 1 except for English General Ability. In gra^e 2 nean gain differences 
vere approximately ont-half of a atsndard daviation unit for tha English tests 
and at leac< one standard daviation unit for the SpaQ|th testa. In grada 3, oean 
gain differences vere at least one-half a standard deviation tmlt except for 
English Reading. 

Suasary . Although exact statistical tests of sigiilficance are not' available 
shoving Dtan gain cosparisions for prograa and control students, the nean differ- 
ences vlth fev exceptions favor the bilingual pupils. The size of the oean dif- 
ferences are of obvious educational significtoce; that is, there is considerable 
non-overlap in the gain distribution for the tvo groups. 

The procedure for selecting controls^ at^tbe kiiSkergarten level is probably 
acceptsble, given that no real catching 'Sf^^ possible. Controls should be 
better in oral English at the entering kindergarten level. Thereafter, progran 
effect can be revealed in tvo vays. First, through Increased gains during ^ch 
school year for the progran participants over the controls, and second, through 
superior posttest performance for the progran pupils at each grade level. The 
data bear out both of these types of progran effect Indicators. Test adtslnistra- 
• tion order, vlth bilingual pupils adnlniatered the English version of the Incer- 
Aaerican Tests before the Spanish version, and vlth control pupils adnlnistered 
the Spanish version before the English version, vould tend if anything to Intro- 
duce practice effects into the Xnter-Aaerican English version scores for "the 
control students.^ 

,'With respect to achievement gain coaparisons. Interpretation is clearer 
if xhe groups to be coopared are equivalent at pretest. This is not possible 
in this situation. The controls are ^*nost equivalent" (to the extent that is 
possible) at the kindergarten level. After that, nean pre- ancf posttest, as 
veil as gain conparisons, overvhelningly favor the progran pupils as shovn in 
Tsble 5. 

TABLE 5 

Mean Gain Conparisons Favoring the Bilingual Progran Pupils 



Grade 






1972-73 








1973-74 


>, 


Overall 


Pre 




Post 


Gain 


Pre 




Post 


Gala 


K 


1 of 


2 


1 of 2 


0 of 2 


1 of 


2 


2^oI 2 


2 of 2 


>»of 12 


1 


' U of 


U 


U ot U 


3 of A 


4 of 


U 


4 of 4 


4 of 4 


23 of 24 


2 


3 of 


U 


4 of ^ 


U ot U 


A of 


U 


A of ^ 


{ ^ 


23 of 24 


3 


2 of 


U 


3 of A 


4 of A 


3 of 


U 


A of A" 


X of u 


20 of^4 


U 










2 of 


U 


3^f A 


Coi U 


9 of 12 


Totals 


of 


lU 


12 of lA 


11 of lA 


.lU of 


18 


17 of 18' 


18 of 16 


82 of 96 



J. 
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PaOCRAM AREA: 



Bilingual Education XSpaaish/Eaglish) 
PROJECT TITLE : 

Bilingual Educacion Prograa (A^rendeaot En Dot Idiooaa) 
LOCATION ; " • 

Corpua Chriaciy Texaa * 

SOURCES AKD LEVEL OF FUXDIKG ; > ^ 

,The following asounca were allocated for the fourth year of operation: 

Title VII $ Jl.OOO* 

Title I ^ , 34,600 

Local 17,202 

MinlEua Foundation Prograa 180,000 »* 

Total Year 4 Allocation $322,802 

^Actual axpenditurtt froa the, Tltlt VII allocation were $82,202. 
.<«^*thia sua includts atctt and local Doolta for ttachtr aalarita. 

PROGRAM START DATE ; ^ h 

1970 * ' ^ ' 

BRfEF DESCRIPTION OF PROJECT : ^ 

Coals and objectives . The Corpus Christi Independent School Distji^t has 
alaed at providJ.ng an effective bilingual-bicultural program of instruction for 
children of Kexlcan-Aaerlcan descent. The District has;^chosen four ways of 
reaching chat goal: (1) the use of a set of instructional strategies and. routines 
selected specif itally for the {>rograa; (2) the review, selection, adaptation,* and/ 
or develppoent of appropriate materials; (3) the ipplcmentation of a staff develop- 
ment prograa tailored to prograa strategies and teacher needs, and (A) the direct, 
extensive involvement of parents through voluntary participation in a variety of 
prograa activities that depend on their help# - ^ 

Context . Corpus Christi is a city of over 200,000 people on the coast of * ^ 
Texas. It la supported by agriculture, petrochemical and netal Indus tries - an^ 
manufacturing plants*,, but Is also the mercantile center for much of the southern^ r 
part of Texas. Tt>e Port of Corpus Chrlati, one of the nation's top 10 harbors;^' 
it Just next to the downtown business district. The population is Bobila, partic- 
ularly cha portion of Kexican-Aaerican descent. 

In 1974, the school enrollment in the Corpus Christi Independent School Dis- 
trict was about 44,000, of whom 54Z were Spanish sumamed.- In the* project, however, 
91% of the children have Spanish surnames. .Recording to parents of participants, 
about lUX of the children are nat'ive Spanish spealcers. A teacherV^rvey in the 
three project schools revealed ^hat about half of the children relied most on their 
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oAtive SpAnish l^hgu^e for cocsunicacion; cbe ceacher opinionB vere verified by 
adalnlttcring the Oral English Proficiency ,Te8t <21i2^cth ^Ott) to a lOZ raadon 

acaple of project population.^ > 

• *. 

•Aaong faailiea whose children are in the project, BIX receive incomes with- 
in the poverty level. The average fanily haa 4.5 childreni. The abaentee rate 
for the diatrlct ia high, particularly in kindergarten claaaea. ^ 

t: 

Hajor featurea of the prograa .. Key prograa atrategies which were evident 
fr6s oa-aite obaervation and progras docxoientation were as follovs; 

• Uae of bilingual teachers exclusively at Kindergarten and Grade 1; 
^ where claases cust be assigned to monolingual teacbera at Grades 2 

through 4, pairing thca with bilingual tAchers for cooperative 
teaching . 

. Providing teachers with weekly visits and personal help f roo bi- 
lingual Instructional Consultants who have classrooa experience in 
bilingual education 

• Daily, cooperative planning anong teachers and aide^by grade level 

« Careful planning, coordination, and oonitoring of the prograa by 
adninistrativa staff 

• Introduction of English reading readiness and aural-oral skills aa 
aoon aa a child deaons traces ninical aural coaprehens'lon 'of English 

^ Eaphaais on English reading and coccunication 'akills for the duration 
of the prograa 



• . Use of bilingual teacher aides for instrftctionkl support and rein- 

forceaent \ 

• • Use of teacher-evaluated and teacher-developed curriculum aaterials 

tailored to participants* needs 

« Providing intensive preservice j;.raining for all project staff, rein- 
forced by monthly inservice days during the school year 

• ■ Promoting parent comltnent to the prograa through their direct in- 

volvement in many ClassrOota-related activities 

Each of these major prograa features will be explained more fully in the sections 
that follow. 

Grade levelg, years of operation, size . During* its fourth year of operation, 
1973-74, the prograa served 519 children in grades K through 3« 

Staffing . The program staff required in 1973-74 for this number of partic- 
ipants waa aa follows: one full-time Project Adainisbrator, one full-time Instruc- 
tional Consultant/Parfint-Comaunity Involvement Coordinator, one full-time Evaluator/ 
Staff Development Coordinator, 22 full-time teachers, And 15 full-tine teacher aides. 
<Eight'"^of the aides served the eight kindergarten teachers fulltlae; the other seven 
aidea divided their time Among the 14 teachera in grades 1 through 3 claaaea.) 
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Ail ataff were bilinguftl. In 1974-75 » yith.the expansion of thfe-progrjua thtough 
Grade A« chree monolingual ceachert were added co'^'t}{e acaff to naintaia .itr'ethQl<s*.> 
balance aaong inscrucCora chac oacche^ Xhe echnic balance of che project schools. 
K second fiill-cise Inscructlonal Consulcanc vas also hired. The ?rograa Evaluacor 
Is nov parc-clse In order Co free salary funds for che second Instrucdonal *Cdnr 
sulcanc. 

^ Scaitf roles In prograa nanageoenc — To help- ceache^s ioplcscat che pr or 
graa properly, an Inscrucclooal Consulcanr vlslcs every prdjecc classrooa a^^^the . 
chree parclclpaclng schools ac lease once every veek and gives Inscrucclooal supporc 
in a varlecy of vays, e.g., procur.eaenc of Inacrucclonal oACerlals, ^^slsCance in 
working vich individuals or groups, deaonacration of teaching cechni<tue8 vlth indir ' 
viduals or groups, advice on individualizing inacruccion. The loscructional Con- 
sulcancs have several years of experience ctachlng bllingually and feel chia has 
helped chea co be effeccive and co escabllsh rapporc vich che children and cheir 
ceachers« ceacher aides, and parencs. • v 

The Program Adniniscracor also visits classes roucinely and info^ially,' 
and drops in occasionally co join che ceachers' planning periods. His special 
concern ia aaincaining good rapporc between che ceachera and che principals and 
becvean trr^ncipala and progran adolniscracive staff. 

Teachers ac each grade levc^l oe^c four days a ve^ during the children's' 
physical education period to coordinate lesson plans. Where oonolingxiai and .bi- 
lingual teachers trade groups for English and Spanish reading, language arts, and 
aath, additional coordination beyond the daily planning period oay be required. 
Cooperative teaching also occurs aaong bilingual teachers, who may trade^ groups f^or 
instruction in various content areas. Thus, the daily planning period is essential 
to track progress and sake plans for 'adjusting the instructional program for each 
child. Teacher aides are frequently involved in these planning aessions. 

Curricula, materials, and t Ipe involved. . 

Kindergarten -'^ Children are not screened for eligibility, and participation 
in the program is voluntary. During the first weeks of school, kindergarten teachers 
closely observe children in class and at play to determine the extent to which they 
rely on their native Spanish language for coomunication. Their ability to comprehend 
EnglijBh is one basis for grduptng and regrouping children for the remainder of the 
year. , 1 . 

In all project kindergartens, nearly .two hours every day are devoted to lan- 
guage ^ eve lopoant and language arts In English and Spanish. For example, 45 minutes 
in the morning are used for English and Spanish language developaient during a ''shar- 
ing experiences" period, discussion of the day, date, weather, and other calendar- 
related events, and writing on the blackboard stories the children dictate to the 
teacher. Another hour in the afternoon is reserve^ for auralr^^ral «nd reading readi- . 
ness skills in English^ and Spanish, using very detailed teacher guides developed 
especially for the program. In both languages, the patterned practices stress b^ic 
sentence patterns and illustrate changes in word forms and word order. Correct pro- 
nunciation and intonation are stressed during these drills; In contrast, during the 
times when free expression is encouraged,^ teachers do not correct the cluldren or in 
any way evaluate their performance. The emphasis at these times i^ on creating excite- 
ment and adding fuel to the children's growing enthusiasm ^or conbunicating in Eng->- 

lish and in SpSnish. 

• lJ_ I 

I English reading readiness is introduced as .kindergarteners are able to learn these 
skills. For SOX of the children, English reading readiness skills are taught in kin* 
dergarten; for | the other 20Z, English readiiig readiness skills ai^e introduced early 
in Grade 1. | ^^^/ 
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' Ocher periods of ciae are sec aside for activities >^ch lend tbe»- 
aelves to further developaent of coscuaication skills. For exaaple, nuaber 
concepts, story tioe, filttstrips, susic rhythss, snack tiz&e, and cleanup are 
used to develop cocissunication in Englfsh and Spanish. Even during independent 
work-play activities, the teacher and aide circulate to. provide each .'child vlth 
at least a brief opportunity to dlacuss his activity in both languages.' Thus, 
froct the beginning, Che progtan eophasises cocsunlcation skili;i heavily, and pro«- 
vides for their systenatic developaent in Spanish and in English throughout the 
daily schedule of activities. ^ 

Cooaercial a^teriels used in addition to the specially prepared Spanish 
and English aural-oral *guidea are reading reedine^s naterl&ls by Boughton-Mlf f lin 
and HarcourtcBrace Jovanovich, Inc. KatheMtics textbooks are by Addison-Vesley 
(Spanish edition) and Boughton-Klf f lin. 

For cost kindergarteners, instruction is provided In Spanish foi 90% 
of the day and in English for }fiX of the day. Individual differences snong chil- 
dren necessitate adjuscaents in language eophasis, and by the end of the kinder- 
garten year, oany children are receiving instruction in the two languages for 
equal aaounts of tine. 'Before the end ^of Grade 1 this is true for all children. 

Grade 1 — Each day about one hour is devoted to Spanish reading'^ and 
language arts, and about two hours to English reading and language arts. -The 
tifiit en individual child spends on these activities fluctuates depending on his 
"need. Other significant blocks of tioe are scheduled daily for Spanish social 
studies (enphasis jon'Spantsh culture and heritage) and' f or,^the&atics — about 
three' quarters of an hpur.each. 

Phonetic analysis skills ^re taught first. In Spanish because of the 
highly phonetic nature of the language. The teachers use detailed lessons and 
activities specified in the specially prepared Spanish reading guide. The guide, 
divided into several levels, also specifies charts the teacher oust make to ac- 
company the carefully sequenced lessons. In the early lessons on vowel sounds, 
xhe guide also presents dialogues the teacher uses to get the children to talk , 
about the sound and to practice saying it and recognizing it in oral speech. 
Each lesson in the guide also suggests suppleaentary activities the teacher or 
aide can use to reinforce the lesson. After the children have learned the vowels 
and can recognize them as initi&l sojunds or as sounds within wo]pds, they learn 
consonants in the same fashion, according to sequenced lessons in the guide. 
Lessons then progress to sounds of letter combinations and syllables. By the 
end of eight to ten weeks, every child will have completed the unit on phonetic 
analysis sk^^s and will have begun to decode words. Other reading skills are 
also taught through lessons in the Spanish reading guide »- Once children Mre 
ready to read in Spanish, the Laidlaw series is used. 

» * * 

English reading readiness akills for Grade 1 youngsters are taught 
through the Harcourt ^Brace Jovanovich, Inc. series. Phoneti-c Analysis skills 
are intfro^uced after the child has learned thoa'^through his 'Spanish 'reading 
lessons r Teacher guides accompanying the commercial Bnglieh reading ieries 
provide instrvctions for conducting the dai]|.y lessons. There is a heavy phonics^ 
emphasis » with comprehension skills receiving heavier acsphesis as decoding skille 
develop* Other commercial ^materiels used foa English reading are by Houghton- 
Mifflin (Listen and Do) ano by Scott Foresman (Talking Alphabet). By the last 
six weeks of the school ycisr, the top reading group in each firat grade is 
reading second grade books in the Harcourt Brace Jovanovich » Inc. series. 
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In Grades 1 chrogis^ nache&AClca la taxighc using , the AddisoQ*47e8ley 
series in English, but Co help provide Inacruccion in Spanish as veil, the teacher 
ilso uses a ccepanion ceachar*a edition of the aeries in Spanish. In Gradea 1 
through ^ the teacher-developed Spanish social studies guide is supplcaented vlth 
lessons in English social, studies froa a textbook by Barcourtt Brace, and World. 

Daring the tlae devoted td theae subject areas, English and Spanisli 
say be used interchangeably. There la no !regl&en for this, and teachers use 
varioua atrategies; in general they switch back and forth between the langxiagea 
depending on the difficulty the chl^(^ren 'are having with the concept or subject 
setter, or a teacher will devote one week to eaphasizing initial Instruction in . 
Spanish with reinf orcecent of the concepts in English, then one week to the pp- 
ppsite order of presentation. Befbre the end of Grade 1 English and Spanish are 
being used equally. 

Grades 2, 3, and 4 — In Grades 2 and 3' Spanish' reading continues to 
be taught according to detailed lessons in aore advanced levela of the ape^ial 
Spanish reading guide, and the Laidlaw aeries continuea to be uaed through Grade 
4. The Harcourt Brace Jovanovich, Inc. series is used throughout for English 
reading. Although leaa tine is devoted to reading and language arts in both 
languagea than was true of Grade 1, these skills continue to be emphasized in 
Grades 2, 3 and 4. By the last six weeks of the year, children in the best read- 
ing group in each ^lass are reading 1>ooks on the next grade level. 

Grouping strategiea and individual attention> Gradea 1~4 — To increase 
the effectiveness of instruction in both languagea in the elementary gradea, chil-^ 
dren are grouped for activities in the various subject areas according to language 
doainance and achleveaent. There are uaually three or four groupa working aepar- 
ately ia every classrooo, on^with a teacher, one with her *aide, while the'other 
one or two groups work on independent asaignsents oade at the beginning of the 
day. Groups ^are.ssall, usually seven or eight cTiildren. The groups rotate every 
20 or 30 minutes so thst each child receives a substantial portion of instruction 
in each subject from the teacher and reinforcement frco the aide. In addition, 
children who need^extra coaching in reading and language arts receive special, 
individual help from the teacher or the aide while the rest of the class ^is work- 
ing on independent seat work, proji&cts, or dsily assignments. At least on'e ^ 
psrent is usually in the classroom to listen to children /ead sloud, to keep ^ ^ 
eye on youngsters who are working independently, and to chat in Spsnish with tliem' 
about their activities ind interests. 

After-school study centers,- Grades 3-4 — In 1974-75, program participants 
in Grades 3 and 4 had acceas to further individual help from one of the project 
teachers for one hour after school at each of the participating schools^ Up to 15 ^ 
children from each project class of thir^^and fourth graders were accepted for this 
hour o{ personalized asaiatsnce in problem areaa in reading, language arts, and) 
mathematics. ^ ' *' ' 

. . ' ;■■ - 

- facilities . No .remodeling of existing classroocl facilities was requix;^ to 
implea^tv the program. In one project school classes^ sre self-^contained. In the 
second, Sc};ool, all classes at a grad^ level are conducted in an "open apace" large 
enough f oi^. t^eachers to instruct their tlssses in different areas of the room. Re- 
grouping'.-aiiid cooperative teaching (dsily trading of groups and teachers for par- 
ticular subject sreas) is very convenient under this srcnitectursl plan. In the 
third project achool, the building * le* dlrculsr, with "pie-shaped" classrooms having 
access .to s central resource center. 
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?reservlce/laservlce training . A week of concentrated preservlce training 
required for teachers and teacher aides prior to the beginnidg. of school eAch year. 
During the corning, all staff l\ear guest speakers 5.roo universities lecture on 
areas related to project philosophy, objectives, and Insrtructlonal procedures. 
During the afternoon the speakers act as consultants during workshop sessions ln» 
^ich tetchers spply principles and procedures presented during c)>e lecture. In 
^hwe sessions, teachers share IdeAS, develop cater lals, deaonscrace techniques, 
and discuss problca areas; ceAchers are usually grouped by g^ade. level for xhese 
specialized sessions. All project teachers and" teacher aides. *re paid for partly 
cipating in the' preserylce, training week, and there has been lOOX participation 
every year. ^ ^ ^ ^ 

Dxxrlng ^he year one day of inseryice tra^ining is required for teachers every 
nonth during school hours; substitute teachers cover project classes on these 
days so that participation o| ^1 teachers is possible. The inservice sessions 
deal with interests tet^heVs indic^fe to the project adalnistrator and Instruc-r 
tional Consultants and provide ftirdher opporcunitiea for idea sharing and refine- 
oent of Instructionsl procedures. ^ 

Every preservlce and inservice session is critiqued by teachers and teacher 
aides on special rating foras. Their* feedback is used to codify future training 
Sessions and to plan addixional programs of special interest. 

Another feature of staff developeent is the use of the Flanders Interaction 
Analysis, with flew teachers, /ideo^pes ar^ nade of classrooa lessons in Septesber, 
January, and Ma9. The project eval^tor analyzes the tapes and confers with each 
teacher to review and discuss the findings. These conferences are private, and 
both negative and positive aspects of the tapes are reviewed.. The teachers are 
guided to define their own concept of i?leal teacher behavior and subsequently to 
qodify their own behavior in the direction of that idea. Since the focus of the 
project is on language developoent i^tpsrticul^r attention is pa^d to the percentage 
of student talk md of indirect teacher approach. 
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Parent^cocnunity involvcaent . Pour to five parent oeetings are held yat e*^ 
• of the project schools every year^ led by one of the Instructional- Cons u^ian^ 
Tofics for thtse neetings include parental involveaent ac.hoae and school, ^^(aching 
aides, tht Toy Lending Library, and the goals and objectives for the projedt^ 
Through' thtt-t neetings and through contact with the teachers* tht parents are 
•ncouragtd to htlp with classrooQ activities. They help to reinforce -acadeaic 
work,, supervise plsyground plVi chaperone field trips, bring snicks, tell stories, 
make charts and other instructional aids, and talk wieh chi^dreti inforoally in 
class.- Parent involveaent has ^ impressed drinatically since the prograa began — 
froo nearly total noninvolvement to 4^nating over 3300 hours of help in clsssrooa* 
. related actj^vities in 1973-74. A Parent Advisory Cqooittee (PAC) is elect^ each 
year and consists of 10 parents, tl^ project adainistratpr , and the Instructional 
Consultants. In 1974-75 there were 5 parents froo one»8chool, 3 froo the second 
sch6ol^ and 2,froa the third, representing Grades K through 4. The PAC^^oeets once 
a Donth to hear progress reports on the progran, to discuss what the prograa 
should be accoopllshing, to review grant appllcJil^ionsV and' to discuss ideas for 
nodifications in the progran. One idea that was IneiCft'porated in the 1975-76 
grant a;^pllcatlon was preservlce and Insetvlce training for parent volunteers." 

Cost . Budget InfotTiation provided by the project adoinistrator was based 
on Tltlt VII allocations end expendUj^ts for four years. For tht first thrtt 
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yeATf of progrm" operttlon, this taouated to nearly lOOZ of the budgets A 
sxssAry of total expenditure*, enrollneatr and associated per pupil costs for 
four yearif it given belov. ^ 







Enrollment 


Total 
Expenditures* 


Per Pupil Cost 


Year 


I 


U3 


' $65,460 


$458 


Tear 


2 


290 


$60,265 


^208 


Year 


3 , 


435 


$8^,025 


$198 ' 


Year 


4 


519 ^ 


$82,202 


$158 



^ *Excludes teacher salaries ax^ salaries for teacher aides 
in T-^tle I schools^ only Title VII expenditures are shown.. 

In oMer to coopare per pupil cost for the total educational progras of 
children in the Title VII project vlth the cocparable figure for children in 
the regular prograa, further calculations vere aade. For 1973-74, based on 
total, funds allocated for the prograa by federal, state, and local sources 
(page 1 of this suscary) , the per pupil cost for the bilingual prograa was 
$622 for ^ each of the 5l19 children served. A finance officer for the Corpus 
Christi School District indicated thst the codpsrable per pupil allocation in 
1973-74 for children not enrolled in ^e bilingual project was $541 per child 
in the tleaentary gradea. Both of these per pupil cost figures iMlude all , 
allpcations for the total educational progrca for these two group^f children, 
except anounts budgeted for *^intenance of operations'." 

, ' The drop in per p«^il coats over the first three years, when Title VII 
funds covered alsost al^ prograa costs but instructional salaries, reflects 
the gro%d.n^ esphasis on carefully .reviewing expenditures for travel, materials, 
and •e4uipaent and on sharing available resources. A najor expense has been ^ • 
paying teachers to develop the Spanish and English aural-oral kindergarten 
guides, the Spanish social studies gui/ie, and the Spanish reading guide. Levels 
I through VII. Another najor. expense each year has been paying project teachers 
and aides to attend the preservice training week, and paying substitute teachers 
to cover classes .for project teachers on inservice days during the year. 
EconoQies have resulted froa having parents repait instructional oaterials and 
aids' and cake teaching aids. 
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EVIDEKCE Or EFFSCTIVEKESS: ' . |» 



The ev4lu«cioa (Uca suaearlsed in cbls secddn vlll be based upon the 
evaluitioa rcporta for the 1972-73 and 1973-7A acadeaic years. The data strong- 
ly sugg^at that the bUiagual arogras in Corpus Cartsti results in 8u?erix>r 
^achicveacat- in both languages by project puj>il« In cocrpariaon vith controls. 
It is useful, to suvsarise the results by grade level for the tvo years. 

1972- 73 

• Kiadergaxten . Project pupils showed significantly higher adjusted 
pcitttest scores on the Inte|^Acerican General Ability Spanish 
sul)tests than* did controls. There were no significant differcncea 
on the tvo Inter-Asarican English subtests nor on three subtests 

, of *the Stanford Early School Achlcveaent Tests. 

.\ » 

• First Grtde . Project pupils, shoved significantly higher adjusted 
posttfcst scores on both subtests of both the Spanish and English 
versions of the Infcer-Aaerican Rcadl^ Tests, and also achieved 
significantly higher grade equivalents on all three subtests of 
the SRX Achievement Tests. > 

• Second Grade . Project pupils shoved significantly higher adjusted 
posttest scores on both subtests of both versions of the Inter- 
Aaerican Reading Tests and on tw of the three subtests of the SRA* 
Achieveaent Tests. 

^ • Furthermore » vrith but one subtest exception, the coaprehenaion part 

of the Spanish version of the Inter-Aserican Reading Teat, length 
of cnroilacnt In the prograa was shown to be significantly related 
to test perforaance. 
* * 

1973- 74 

^ Kindergarten . Project pupils showed significant gains in coaparison 
to controls on the Sp ani sh version of the Inter-Aaerican General 
Ability Test, but were equivalent to controls on the Stanford *Early 
School Achicvcaent Test. Objectives were approached but not reached 
at the kindergarten level. 

• First Grade . First graders in the project outperforaed controls by 
approxlnately 0.5 grade equivalents on the SRA Achieveaenr Tests 
(significance levels are not reported), but did s4.gnlf icantly odtgain 
controls on both versions of the Inter-Aaerican Reading Tests. 



Second Grade . Project pupils outperforaed the controls with respect 
to average grade equivalents on the SRA Achicveaent Series (though 
the aean differences were not statistically significant). Project 
pupils significantly outgained controls on the Spanish version of 
the Inter-Aaerican Reading Tests. The reverse was true on the 
English version of the Inter-Aaerican Reading Test (controls signifi- 
cantly outgaine^ project {fupils) . 
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rnlrd Grade , While all differences on the three subtests of tfee 
SEA Achiev€saei?t Tests favored the project group, there yere aone 
si^ificAnt. . However, project pupils outgalned the controls on 
both versions of the Inter-Aserican Reading Tests^ 
/ 
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Tne Title VIZ 5iUnguai Education Program in Corpus Chrlsti, Texas is in the 
fifth year of operation. During the 1973-74 scadealc year, the l4st full year £or 
which evaluation data is available, 519 pupils were'enrolled in the project i;i 
grades K through 3 in e^ch of t^ree school? (frockett, 2vaas, aad Travis). 
Table 1 shovs the grovth of the Title VII project during itsflrst four years; one 
grade level was added each year. * 

Table 1 



Nuaber of Students Lnr^>lled at Each Grade Level 2ach 
Acadeslc Year in the Corpus Chrlsti Bilingual Education Prograa 



Year 


K 


1 


Grade Level 
2 


3 


Totals 
Enrol Isent 

— < 


1970-71 


U3 








143 


1971-72 


184 


106 






2^0 


1972-73 




155 


107 




435 


1973-74 


•154 


148 


- 124 . 


93 


519 



Evaluation 

— « ' 

The Title VII evaluator in 1972-73 and ip 1973-74 was Ilene Cordray. 
She is al8(< a staff neaber of the Corpus Chrlsti School District Departnent of 
Planning and Evaluation, directed by Dr. Ronald Howard in 1972-73, and by Dr, 
Kenneth Kyle in 1973-74. 

P^pil Characteristics 

In 1973-74, the total C.C.I.S.D. student enrollocnt was slightly under 
4^,000 with approxlnately 32,5QO Mexican -Aaerican pupils (54X), 2,400 li3,ack 
pupils (5X) and 17,700 Anglo pupils (4i:). The 519 Title VII project pupils 
were 92X Mexican-Aaerlcan, 6% Black and 2X Anglo. Two of the three -project 
schools show sinllar ethnic balances with the exception of the Evans school 
with an enrollBfcnt of 75t Hexican-Aaerican, 24t Black and It Anglo. 
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Data reported for the 1972-73 acadcalc year indicated lov «ver*se noathly 
faaily ^nco=« levels of $356 for project earollees vlth 81X of the pupils', fasily 
^iacoses falling below the poverty level. Aver agi^ f tally stxe included 4.25 
children based oa parents* reports; 323 pupils i74Z of the 435 enrollees) were 
native Spanish speakers. Average class enrollaents in Sepcesber 1972 vere 27.75; 
vltbdfavals during the 1972-73 year sveraged 4.81 per class (17 per cent); late 
entries per class averaged 4.00 <14X). Absences during the year vere highj 
sveraging spprozlaattly 353 per class or 12.7 par pupil. 

Three eleaantsry schools selected by the Title VII staff sarved aa 
'control frchools. One class at each grade level -vas selected vlthin each of the 
three control schools by the principal in charge of each. Classes foralng the 
control group vere spproxiaately equal to the project group in terms of socio- 
econo:zic levels Each control school vas also the one closest geographically to^ a 
project school. 

Data reported for 1972-73 also indicated that kindergarten teachers in the 
control schools sveraged 20 years* teaching experience, while the sii^ kinder- 
garten teachers in the project sch6ols vere all in their first or second year ^ of 
teaching. Class size for control kindergartens sveraged 23 in comparison with • 
an average size of 29 for project kindergart^ classes. Otliervise, at grades 
1 and 2, project and control classes were* equal in terss of teachers* experience 
and class size. 

Until the'current 1974-75 acadeaic year, classes in the control schools 
vert taught zaonolingually without regard for the language dominance of the 
children. Vlth the icpleaentation, of Te^s bilingual progras legislation in 
1974-75, pupils in the control classes say also be partially instructed in 
Spanish. The project cvaluatdr reports that in the .control schools, heavy 
emphasis was placed on tSe instruction of reading an^ cathesatics just as in 
the Title VII bilingual education progr&n. ' 

Comparison Design 

The basic coaparisoa design in 1972-73 and 197>-74 involved adiainistration 
of pretests and posttests to both project and control groups within each grade. 
Tha evsluators point out that tha tvo groups say not show saspling equivalence 
although the atteapt vas aade to balance both, groups 'as closely as possible in 
tarts of ethnic' aix, aaturatlon, and the test instrusente administered. Match- 
ing vas Bade at the school level. In this situation, if pretest equivalence is • 
observed for the controls and project pupils, then significant aean posttest 
scorjis, either rav or adjusted for pretest group differences, or significant 
seen gain scores favoring the project pupils indicate program effect, other 
things being equal. Data vill be reported for the 1972-73 aiid 197^-74 academic 
years^ > 

Measured 

I — 

For the 1972-73 and 1973-74 academic years, the folloving tests vere 
administered at the grade levels it^icated. 
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Gr4de« adaihlsf rftd In: 



Tc»t 1972-73 • 1973^ 

Stanford E^rly School Jlchicvcaent Te«c K ' t < 

Xncer-Aaerl&tn G«ner<I Ablllcy (Sptnlsb)' ^ t K 

Intet*-Aaeric«a General Ability <Eagli8h) t * t 

SRA Achievcacat Te»t (Prlsary' I) * 1 1 

SRA Achievcaent Teit (Prlsary II) • 2 . 2>3 

Incer-Aaericaa Reading (Spanish Level I) If 2 ' 1»2 

Incer-Aaerican Reading (Spanish Level II) - 3 

Inter-Aacrican Reading <Eagliah Level I) 1,2 1,2 

Inter-Ascrican Reading (English Level II) - 3 



Results: 1972-73 

Kindergarten , Table 2 shovs neans on the Stanford Early School Achievcaen£ 
Test (Kiy administration) for the kindergarten project and control pt^lls enrolled 

during the 1972*73 acadealc year* Table 2 alao shows oaan pretest, posctest, and 

posttest-*adjusted-for-pretest group saan differences on the verbal-nuaerical and 

non-verbal parts of both the Spanish and English language versions of the Inter- 
American General Ability Test*. 

In kindergarten, both the cbntrol and project group shoved equivaljent 
t^eans on the three subtests of the Stanford Early School Achieveaent Test 
and on both tests of the English versions of the Xnter-Aaerican Test of 
General Ability. On the Spanish version of the Inter-A»erican (Pruebas de* 
Habilidad General), the adjusted nean posttest differences significantly 
favored the project pupils on both subtests* In both English and Spanish 
language developaent the project* pupils scored above the 64th percenslle on 
both language versions of the Xnter-Aaerican Test, while the controls scored 
significantly lower (at the 46th percentile) on the Spanish version* 
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7able 2 



- Hetnt on the SCAnford E^ly School Achleveaenc Test and on 
, the Intcr-AsericAu Ctt^crai Ability Test (Sp«ni«h and English 
Vcr«ion«)^for Bilingual Project and Control Group Students 



*Grade 

> (Language) 



Group 



Stanford Early School Achieveaent Test 
Kacbezatics 



Letters, 
Sounds 



^ Aural 
Cosprehension 



iCindergaxten 
(Soglish) 


Project . 
Control 


167 
67 


17.4 17.6 17.0 
18.6 18.0 18.0 








Inter-Aaerican Pruebas de Babilidad General 








Verbal-Nuaerical Kon-Verbal 








Pre Post Post (ad j) Pre Post Post (adO 


Kindergarten 
(Spanish) 


Project 
Control 


59 • 

♦ 


9.6 12.9 13.0^^ 11.5 15.2 15.2. 
10.3 12.1 11.8 12.1 14.5 14.3 








Inter-Aaerican Test of General Abi3.it_y* (English). 








Verbal -tiunerical Non-Verbal 








Pre Post Post(adj) Pre Post Post(adj) 


Kindergarten 
(English) 


Project 
Control 


59 


10,4 14.4 14.3 12.0 15,1 15.1 
10,2 14,0 14,1 11.0 15,3 15.3 



5 

* Mean differences between Project and Control group at ,05 level 

/ 

** Mean^ differences between Project and ^Control group at ,01 level 



Raw score oeana are shown for the Inter-Anerican General Ability Test. 
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First Grade . First gX4de suacarles are sbovn Dd Table 3 vhlch reports 
grade equivalent aeans for project aod control groups on the three .^tescs of 
the $SA Achleveceat Test and pretest, posttest, and gain score tteans on both 
subtedts of both Iangaa<e versions of the Inter-^American Bfsdlng Tests. 



Table 3 ^ 

Keans on xh%. $SA Achievement Test and on the Inter- 
Aaerican Reading Tests (Spanish and English Versicos) 
.for Bilingual Froject and Control Group Students 





9 




SEA Achicveaent Test (Prloary I) GB's 






Group 


N 


Reading ^Language Arts- Katheaatics 






Project 
Control 


146 
75 


2.2 2.1 2.6 
1.8** ' 1.7** 1.8** 





Inter -Aaerican Pruebas de Lecturs (Kivel I) 
Vocabulary Cocprehenslon 
Pre Post 



♦ Project 
Control 


126 
65 


9.3 20.4 20.4 
7.7 9^ 9.2** 


9.5 16.2 16.2 
8.3. 8.7 8.8** 






Inter-Aaerican Test 


of Reading acvel X) 






Vocabulary 
Pre Post PostCadj.) 


Coaprehenslon 
Pre Post Post (adj.) 


Project 
Control 


126 
65 


10.9 23*9 23.4 

8.5 18.0 '18.9** 
i 


9.9^ 19.6 19.4 
7.8 13.6 14.1** 



** Kean differences between Project and Contrpl group at .01 levej. 

Both the project and control groups, perforsed veil st the eod-of-year 
adainistration of the SRA Achieveaent Tests (1.8 " appropriate grade level), 
but the project group scored significantly higher (.01 level) in mean grade 
equivalents thsn did the controls on all three subtests. On both language 
versions of the Xnter-Aaerican Reading tests, the adjusted post test means of 
the project pupils were significantly higher (.01 level) than the means (of 
the control group on both the vocabulary and coaprehension subtests. 



V 



Raw score aeans sre shown for the Xnter-Aaerican Reading Tests. 
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Second Grade . Secoz^ grade sussarles are shovn in Table 4 for Che saae 
tests as those appearing In Table 3. 



Table 4 . 

Means on the SHA Achleve:=ent Test and on the Inter-Aserican Reading Tests 
(Spanish and English Versions)^ for Bilingual Project and Control Group Students 



Grade 

(Language) 

\ 


Group 


K 


SSA Achicveacnt Test (Prlcarv II) GE's 
Reading Language Arts Mathematics 


Second 
(English) 


Project 
Contro'l 


106 
68 


2.9 
2.6 


2.8 
2.5* 


2.7 
2.4* 








Inter-Aaerican Pruebas de Lectxira (Klvel I) 






• 


Vocabulary 
Pre Post ^ost(adJ.) 


CcQprehension 
Pre Post Po8t(adj.) 


Second 
(Spanish) 


Project 
Control 


92 
68 


14.6 26.4 
12.3 16.4 


25.8 
17.2** 


10.3 21.1 . 21.4 

11.4 14.6 14.2** 



Inter-Aaerican Test of Reading (Level I) 

Vocabulary Coaprehension ^ 

Pre Post Post (adj.) Pre Post Post (adj.) 

Second Project- 92 22.4 34.0 33.7 17.7 32.8' 32.4 

(English) Contirol 68 21.3 30.7 30.9** 18.3 27.7 27.5** 

* Mean differences bttveen Project <nd Control group at .0^ level 

** Main difference^ betvatn Projtct and Contro^roup at .01 lavtl 

The project students naintained grada level (2.8) at the Hay testing of the 
^SRA AcMtveaent tests and significantly (.05 level) out-scored the controls on 
the SRje language arts and aatheaatics subtests. On both the English and Si>ani8h 
versions of the Inter-Aserican Reading tests, the project group*s adjusted posttest 
neans were significantly higher (.01 level) than the control's on both the vocabu- 
lary and coaprehension tests. / ' 

r i • • . -— 



Raw score neans are shown for the Int^r-Aaerican- Reading Tests* 
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LADgu^ge Developaea c. Project pupils «c the end of cbe first gr^de vere 
cltssified into chot« who had been in the prograa for cvo ye^rs &od chose vho had 
been in the progrta only for the first grade. On all three subtests of the SRA 
Achleveacat Test— Reading, Langiiage Arts, and Hathcnatics— the pupils in the 
first grsds vhe had also been tsrolled in tha bilingual prograa la kiodergartea 
dcorad significantly hlghar than pupils enrolled for only the first grade. The 
itasa pattern of significant differences becvten SRA Achieveaanc Test i&eans for 
second graders enrolled three, cvo, and one year in the prograa vas observed. 
These data are shovn in Table 5. 

Table 5 

Mean SRA Achieveaent Test Scores for First ^d 
Second Graders Classified by Length of Enrollaent 



Group 



Reading 



SRA Achieveaent Test 
Language Arts , Mathenatics ^ 



First Graders in 

Prograa for: 

2 years 
I yesr 



100 
23 



2-3, 

1-9 



2-3, 
1.8 



2-7, 
2.4 



Second Graders in 
Prograx? fot; 

3 years 
2 years 
1 year 



42 
21 

27 



3.1. 

2.8 

2.6 



3.3^ 

2.5 

2.4 



2.S. 

2.5 

2.5 



* Mean differences b^tveen gjroiips at .05 level 
^* Mean differences between groups at .01 level 
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KtMti pretest and posttest rav scores on both the Ex^llsh aod Spanish 
versions of the vocabulary atvd coQprehenslon subtests of the Inter-Aaerlcan Reading 
Tests vere also calculated and are reported in Table 6 for first graders enrolled 
in tha project for one or Cvo years, and for second graders enrolled one, tvo, or 
three years. In every instance, vith ihe exception of the Cooprehension posttest 
scores for ^irst. graders on the Inter-Anerican Spanish version, length of prograa 
enrollaent was significantly related to aean test perforaance for both grade groups. 



Table 6 . . ' 

• Raw Score Means on the Inter-Aaerican Reading Test for 
First or Second Graders Classified by Length of Enrollnent 



Inter-Aaerican Reading Test 
Spanish English 



Vocabulary Coaprehenslon Vocabulary Cooprehension 



Group 




Pre 


Post 


Pre 


Post 


Pre 


Post 


Pre 


Post 


First Graders 
Enrolled: 




















2 years- 
1 year 


102 

22 


9,8 
7.3 


21«6 
14,6 


9.6. 
9.2 


16.7 . > 
14. 2°-*- 


11,2 
9.6 


25.0 
* ** 

19.4 


10.2 
8.9 


21.6 

13.7 


Second Graders 
Enrolled: 




















3 years 
2 years* 


21 


16.9 
16.5 


29.7 

** 

26.0 


12.6 
10.9 


24.9 

** 

19,3 


26.1 
21.2 


36.0 ' 
** 

32.9 


20.3 
17,3 


'^35.3 
30.0 


1 year 


28 


9.3 ^ 


21.2 


6.0 


16.1 


17,9 


31.7 


14.1 


29.5 



^ean differences between groups at .01 level 
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Reiulci; 1973-74 ^ 

^ Coapariiona bccvccn project and control groups in 1973-74 vere aadc on 

the tase teata used in 1972-73 tP oeasure Enslish and Spanish language abilities ' 
However, poittett covariance adjustsents were not calculated, ao that on the Inter- 
A»«rlcao Sariai teata in both lansua^es, pretest, poatteat, and gain aeana are 
reported, with pro|r«« af factiveneaa deaonatrated by significant seen %Un dif- 
rarancas favoring project 'groups. 

Kindir|artan« Csrtain criteria were set as indicative of the achieveaeot 
of progran objectives for the project group.- On the Stanford Early School 
Achieveaent Test, klndergerten pupils enrolled In project clesses in 1973-74 were 
expected to echieve et as 80/40 (80Z of the pupils will score ebove the 40th 
percentile) criterion and a 40/70 (40Z of the pupils vUl score ebove the 70th 
percentile) criterion. On the Spaniah version of the Inter-Anerican General 
Ability Tes'fc the nean gain of the project group was escpected to show a aignificantly 
(.05 Wei) higher nean gain than the controls with 90Z 6t the pupUs scoring above 
the 20th percentile and 60X of the pupils scoring above the 50th percentile*. ' 

For the Stanford Early Achieveaent Test total score, 83X of the project 
students scored above the 40th percentile and 27Z of the project students scored 
above the 70ch percentile. Mean differences on. the three subtests did not 
significantly favor the project students in any of the three subtests." 

For the Spanish version of the. Inter-Anerican Series posttesf total score, ^ 
85Z of the project pupils icored above the 20th percentile and 70X above t^je 50th 
percentile, indiceting neer attainment of the achieveaent expectancies. 

Teble 7 shows pretest, posttest, and gain oeans on both versions of the 
Inter-Anerican Generel Ability Test for the project and control atudenta. 

* ^- ' Table 7, 







General Ability 




General Abillt7 






Inter- 


"Aaerican Total 


Inter-Anerican Total 








Spaniah 




Engliah . 


•GrOu}/ y 


N 


Pre 


Post Gain 


Pre 


Post Gain 


Project 


'\ 


22.3 


28.8 6.5 


22.0 


30.4 8.4 


Control 


48 \ 


^ 24.2 


26.2 t.\ 


2i.O 


29.3 7.3°*°* 



* Mean differences between Project and Control group at .05 level 

In suanary, objectives were partially achieved at the kindergarten level, 
with pupils in the project significantly outgainlng controls on the Spanish 
version of the Inter-Aaerican Series. 



D-73 




148 



First Grade . Objectives for the first grade vcre.(a) that first grade 
project pupils would significantly outj^in control pupils on both versions of 
the Inter-Aaerican Reading Test, and Q>) that project .pupils would' score 0.6 
grade equivalents higher than controls on the SRA Achieveaant Katheaatics 
and Reading subtests. Suasary test information ^ .prasaotsd in TtSlav ^ ui'd 9^ . 

Table 8 

SRA Achieveoent Test Means for Firsb Graders 

SRA Achieveaent Tests 

Croup' N Read 1 ng * Language Arts ' Mathenatics 

Project 111 2.3 2.2 ' 2.4 

Control * 7A 1.8 1.7 1.8 



^With respect to the second objective, grade equivalent differences favored 
the project group on the Mathenatics subtest at the expected level (0.6), but not 
on the Reading subtest (0.5) of the SRA Achieveoent Test. Significance levels 
are not reported for these nean differences between projact and control pupils^ 
in the first grade, but it should be noted that project pupils averaged considerably 
above the grade level of 1,8. , 

Table 9 



Inter-Aaerican Reading Test Raw Score Means for First Graders 







Inter-Aaerican .Spanish 


Inter-Aaerican English 


Group 


N 


Pre 


Post Gain 


Pre Post Gain 


Project 
Control * 


125 
•46 


19.6 
U.A 


49.5 29.9 
21.3 6.9 


22,2 5Z.9 30.7 
14.2 34.4 20.5** 



** Mean differences between Project and Control gi?oup at, .01 level 



With respect to the total score on both versions of the Inter-Aaerican 
Tests (English^ and Spanish) the project pupils showed significantly higher gain , 
scores than did the controls in the -first grade. 
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Second Cr^de . Objectives for the secoad grade were (&) project students'* 
would average 0.6 grade equivalents higher than controls on the Hathesatics and 
Reading subtests of the SRA Achieveaent Ttst and (b) project students would 
outgain controls on both versions of the Xnttr-Aaerican Reading Tests. Second 
grade project scudtnts outparfonaed the control students on the SRA Achieveaent 
Tests, although the expected differences of 0.6 grade equivalents were not 
observed, as shown in Table 10. 





I 




Table 10 








SRA Achieveaent 


Means for Second Graders 










SRA Achieveaent Tests 




Group 




Reading 


Language Arts 


Kathenatics 


Project 


130 


3.1 

• 


3.1 


2.9 


Control 


83 


2.7 


2.6 


2.5 



Table 11 s^ows mean pretest, posttest,and gain scores for both the project 
and contr61 students on both versions of the Inter-Anerican Reading Tests. Results 
^were nixed with project pupils outgaining controls on the Spanish version and 
•bontrols outgaining project pupils on the English version. 

Table 11 

Inter-Aaerlcan Reading Test Raw Score Means for Second Grade^rs 



Group 



Inter-Anerican Spanish 
Pre Post Gain 



Inter-Anerican English 
Pre Post Gain 



Project 
Control 



97 
57 



33.5 
20.4 



54.0 
35.9 



20.5 
15.5 



44.6 
34.3 • 



^.6 
60.8 



21.0 
26.5* 



differences between Project and Control group at ^.01 level 



Mean 
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thlr<l Grade . Third gr«4e objectives vere Idestic&l to tbose for the secoad 
grade: , (a) 0*6 mttn gr«dt '^ulvalent vu^rlority *for project przplU on the 
Kafheattics'*«ad ,8e»dlri8 tubtcttt of the skA Achl«vc«tnt Tests tnd (b) wun gAln 
^core suptriority for project pupils, on tlSt Inter-Aaerlc^n- -tstHlrtg TMts» MttiXcs 
srs prettntsd ta^bl^ 12 «&d .table 13. , ' i 

' ^ *r , 

* ' * , Table 12' ' \ ^ \ » "' 

' ' ' ' • . " 

jSRA Achiaveafcnt T<sc Keaas for Third '<>if»dtra^ ' ^ - 

• » SEA Achieveaent Tests ] 



Croup * y [ Resding Laagtiagc Arts Katteiarics 

Pr^ect' 87 . ' 3.5 ^3.6 3.6 

.Control 89 .3.1 ' ' 3,5 ' 3.5 



Objectives vith respect to the SHA Reading and Kat&eaa.tics^ subtests v^re 
npt set and grade equivalent levels .for either group do not reach grade expectancy. 
However in r*easding, the project ^pupiis outperfona the ^ptrols by 0.4 grade equiv- 
alents on the avsraga. 



. , Tabl< 13 

Inter-Aaericaa Jteadlng Test JLaiPiScare Keans for Ihird Graders 



Group 


N ^ 


Inter-^Ancrlcan 


Spanish 


Inter-Aaerican English 


Pre Post 


Gain^ 


Pre P^*t • Gain 


Project 


85 • 


' 26.7 56*5 


29.8 


52.5 72.7 20.2 






** - 




Control 


63 


23.8 33.5 


9.7 ' 


48.2 59*1 10.9** 



** ?tean differences between Project and Control group at .01 level 

In the third grade, project pupils significantly outgained the controls on 
both versions of the Inter-Aaerican Series. 
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. Bilinr**! Bducation 

Bilinguar 'EdtK*tilo,a .Prograa (Title VII, ?L 89-10) 
Hoy*jti>n, Texas " - 

The.'follDvia^ figu'rpt^pertain to the fifth year of prograa operation: 
Title VII Tand-t *^246,960" 
Loc^l FunSi- ' ' • 1^2 , '346 
State Fundi .'29^,705 

Total Vear 5 5726^009' 
PROGRAM START SATS : 

1969 ^. _ ' - 

BRIEF D£SCRI?TI0V-07 ERQJ£CT-;» 



Coals and ol>Jf raves.' Thfe.pro&r^ar is designed to serve Spanish speiking 
students by developio« cuUvx-ally and linguistically appropriate curriculua, by 
providing relevant trataing tor ^^acher* siid aides, by developing parent and 
coasunity involv^ent in thfl^sedu.catio.ial process, and by generally effecting a 
change in attitude towards .biilA^l education. The intent of this prograa is 
to iaprove student self^c-Ofnc^^iit-bQ^ raising the status of the students' language 
end culture Ind ^ther*iry .increasing student achieveaent< in English, Spanish, and 
other conteno^arees;. ^ V. 

Context . 1Ji^--Houst5n school district, which is 41Z Black, 40Z Anglo, and 
18Z Mexlcan-Aaerican, experiences a high attrition and nobtlity rate. The 
prograa schools are located in the eastern and northeastern parts of the city 
and have a student enrollment 'which is 53X Kexican-Aaerlcan, 422 BUck, and 5Z 
Anglo, Asian, and* Indian. \ 

Prograa description . . 

^ Gr*d.e level(s)> yes'ra " of operation, size— During its fifth year of ' 
operation, 1973-74, zhit progtan served 1550 K-12 students in eight elementary 
schools,^ ot^e junior high, and one high school. 

Stamn£-«^The prograa staff required for this nuober of participants 
consists of the following; Adainistrators/supervisors (5, full tlae), Specialis 
(1, pArttiae), Evaluator (1, parttiae), teachers (48, full tine) teacher aides 
(23, parttine), and clerical (3, fuU.\loe) , All adninistrators., teachers, and 
aides in the prograa are bilingual. 
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Curricula, material, clae involved — (Tnls section applies to the 
eleaeniary level prograc.) Students begin It .ttuiu^ to read in Spanish vich 
the ute of His Prineras Letras . a reading readiness book oo letters' and letter 
sounds in Spanish. Spanish Reading Activities , a workbook developed by teachers 
in the district, is used in conjunction with the reading teact. After students 
aaster Spanish decoding skills using Kis Prlaeras Letr^s and Spcaisb Reading 
Activities , they begin reading in the Uidlav basal reading series in Spanish. 
As Students tsprove their reading skills in Spanish, they progress through the 
Laidlav basal readers. 

Spanish dominant students are also instructed in oral English Laxiguage 
development, and EngUsh reading using Basic English Language Patterns for -Spanish 
Speaking Students and the Harcourt, Srace, and Javonovicb basal reading series. 
English dominant students are instructed in oral Spanish, Spanish reading (for 
those who are ready), and English reading. 

Each bilingual clas8roo= has a teacher and a half-tiac aide v4io assists 
with instruction. The teacher plans the daily prograa, and the aide works closely 
with her to see that it is carried out. 

Instruction includes a block of tiae devoted to Spanish reading and 
language arts. During the remainder of the day tiie instruction is in English, 
for English doainant and bilingual students. Students who are aonolingual Spanish 
receive additional instruction in Spanish after the lessons are presented in English. 
The bilingual teachers have available to the= the flexibility of explaining, e=pha- 
sizlng, or reinforcing any skill in either language tfi ensure that each pupil be- 
coses involved and senses a feeling of achieveaent and security. 

At the secondary level, the progras includes ESL for Donolingual Spanish 
students and several bilingual courses for students Jho have already attained sone 
degree of bilinguality . The aonol'ingual Spanish students are identified by the- 
hoserooa teachers and tested on basic reading skills. Those students who qualify 
are enrolled in ESL, plus -regular English and siath, and their choice of elective 
courses. Students in their second year of ESL also take Science and History (in 
English). 

Students who do not qualify for ESL 'aay enroll in any of the several 
courses which are taught bilingWlly. These courses are open to all students. 

Bilingual Courses 

Junior High High School 

Reading, Writing, Spelling Algebra 

Texas History Geooetry 

Speech Spanish Business Cocsunication 

Draaa Mexican Polkloric Dance 
Spanish for the Spanish Speaker 

Facilities— The progran can operate in a regular classrooo without any 
special codification. 
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Pareattl Involveaenc — CloM coatect vith parents h*s been 4 key feature 
of the progrxa over the £tve years of its devel^aaent. Each yexr teachers bold 
quarterly aeetlogs with the parents at the school to explaln^ the blljTigual progras*. 
Teasers also z^»j(jc yearly boa« visits to parents to talk about how^t^herr child Is^ 
doing and to obtain the parezvts' reactions to the progrea on a district question-, 
naire. 

Brochures entitlj^ "Parental Involvcaent in the Bilingual Education of 
Houston Independent School Distri/ct" have >een developed by the district guide - 
teachers in getting parents involved in school-related activities. The teachers 
have recruited parents to help on various classrooc projects, and sooe parents 
have tutored bilingual students under the guidance of the teachers. 

Advisory Soard Meetings — During the first few years of the progras's 
development, the district Title VII Advisory Board played an icportant role in 
increasing 'cocsunity involveaent in the prograa. For instance, in order to increase 
participation of Spanish speaking parents, neetings vere conducted in English and 
Spanish. At the beginning these meetings vere used to infom parents of develop- 
oents in the progres. Later , parents becaae nore actively involved in decision 
saking. 

The Advisory Board, vhich aeets (jUarterly, consists of the principal, 
one bilingual teacher, one parent, and one cccsuaity leader froe each of the ten 
participating schools. Tnese sesbers are organised into study teass in six areas; 
Instruction, Curriculum, Cozsunity Relations, los&rvice. Dissemination, and Eval- 
uation. Each teas revievs the needs and/or problens related to its area and sakes 
recosaendations to the total board. 

Preservice/inservice training — Over the first five years of thijprograa, 
teachers gained valuable experience in nany areas of bilingual education. They 
participated in the development of new materials, established their ovn styles of 
dealing Vith the task of ^teaching in tv6 languages, and discovered hov to sake the 
best use of the help an aide could provide in the classroon. The frequent inservicje 
trafhlng. sessions which vere provided by the district for the teachers supported the 
teachers in their efforts. 

This Inservlce training for the teachers and aides in the bilingual pro- 
graa vas provided at least once a month during the school y^ar. Training consisted 
of 2'hour sessions after school for which participating teachers received a $iO 
stipend. During the 1973-7A school year ten sessions vere held, covering the 
following topics: SSL, cultural awareness and history, evaluation> and methodology 
of language instruction. Bilingual staff development activities also included 
visits to- other schools, displays of locally developed instructional materials, 
and nini-university courses offered by the district. ^ 

Cost — During Year 1 the program received funds totaling $23A,850; this 
figure has increased each year by about $100,000 as more participants are served. 
(Program enrollment has increased from 650 to 1550 participants over the five 
years.) The approximate per-pupil cost for the program is less than $500 per 
pupil, based on a five-year average. Major startup costs vere for equipment and 
furniture ($20,000 in Years 1 and 2 combined), instructional materials ($10,000 
in Year 1), Inservlce stipends ($1A,500 in Years 1 and 2 combined), and consul- 
tants vho helped with all phases of curriculum development, inservlce training,, 
and evaluation ($6,000 'in Year 1). After Year 1 the amount spent for consultants 
dropped to $2,000 and continued to drop as local resource people gained experience 
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In the progras tui vcre able to taxe over a coasuitixsg role. As the nee^ for 
ir.a.er%'ice training tapered off, the aasxiat spent on sclpends was reduced. Teachers 
arc paid to vrite carriccius aatertals to suppieaent cocserclal materials, but 
chit activity has becoae les» critical as better bilingual afiterials becooe avail- 
able coBoercially.' 

The Houston Indepeadeat School Dis*trict substantially increased its 
sopport of the progras during Years 3 » 4, and 5 by picking up an progras expenses 
at Kindergarten in Year 3. JCiodergartcn and Grade '1 in Year 4, and Kindergarten 
through Grade 2 in Year 5. ?rogra= staff feel that by assuming fiscal responsi- 
bility for the total progra=» a grade level at: a tiae, year by yeAr» the district 
funds have contributed to the orderly, stable growth of the program. This approach 
is believed to be preferable to one in which a local district piciis up specific 
budget itcss, vithout taking into account sufficiently the i=?act of this action 
on the total progras. 
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tMi>?:NC£ 0? EFF£CnVEK£SS: 

Evt ligation . The Siliagual Sducttioa Project of the Houfltco Independent 
School Dl»t??lct is served p*rt tiae*by the project cvalumtor, H». Piedad Cortez, 
as well aa a part-tise research consultant vbo is responsible for the basic cos<- 
'-parison design. The nain objectives of the evaluation each year vere to: 

• Establish cocparability of progras participants and control 
groups to validate coepsrstivt cognitive gains. 

• Coepare the cognitive gains in general English ability of 
program participants and control groups. 

• Assess gains in the Spanish ability of project participants. 

• When possible, cocpare the sathedatlcs ability of progras 
participants and control groups. 

• Cospare oean posttest general English ability scores for 
control groups aad for program participants categorized 
by years of enrollment in the progras. 

Year and grade sasples . Elementary level test data for three years are 
sums ri red; 1971-72, 1972-73, and 1973-74. These are respectively the third, 
fourth) ai:d fifth years of prograa operation. Data for gfades K-3 are reported 
for 1971-72, grades K-4 for 1972-73, and grades K-5 for 1973-74. In 1973-74 
however, the grade 5 cosparisons are based upon only 36 pupils in the cocbined 
participant and control groups; this number is too s&all to allov strong Inferences 
to be made. Consequent ly« the statistical sijsaries presented in Table 2 and 
Table 3 below are those for grades K-3 in 1971-72 and grades K-4 in 1972-73 and 
1973-74, 

Measures . Measures reported are the general English ability and mathematics 
scores of the Inter-Aaerlcan Series Tests administered \ln both Spanish and English 
to the progr^ participants and in English to the control groups. Reliabilities 
are unreported. This teat is quite generally in use in \lllngual educatl<tn pro- 
grams across the nation. Raw scores are reported. There^are no standardized 
measures, grade equivalent scores, or U.S. norms available. 

Comparison procedures . 

• Pretests were administered in October and posttests in May of 
each academic year. 

• Repeated measures analyses pf variance were calculated, yielding 
tests of group effects (experimental versus control) in the between-subjects 
brealcdovn and main time effects (pre- versus posttests) as well as Time X Group 
Interaction effects in the wlthln-subjects breakdown. The latter one decree of 
freedom contrast is the one of Interest — its significance indicates the rejection 
of the null hypothesis that experimental and control mean gains are eqxial. Mean 
gains for both groups are reported and indicate' in all cases that the estimated 
gains for the experimental group, exceeded that for the control group within each 
grade in each year reported; nearly all gain contrasts were significant. 

• Inferential statistics were F-tests. 
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** ParCiciptnc schools and control schools nuabered seven tnd cvo-^ re> 
•pectlvely, in 1971-72 and the sese in 1972-73; there were eight experiaental and 
two control, schools In 1973-74. Ethnic distributions -of tot4> enrolleeb in all 
project schools for th^ three jears reported were approxiaateiy balanced between 
Black and Spanish-Aaerican vhich resultsd ^oa "pairing'* of schools to observe inte- 
gration; this •'balancing" in Houston was aaong tht predoaiatatly Spaaish-Aaerl can 
• schools. 

^ Control groups are reported as selected on the basis of slsll^rity to 
the experiaentals in language, socio-econoaic level, and acsxiealc achieveaent. How- 
eyer, three factors should be considered when interpreting the results. First, in- 
dividuals were rcaoved froo the control groups in order to satbh the pretest score 
distribution in each grade to the pretest score distribution for the experimental 
group.* Second, attrition for the bilingual participants in the Houston Independent 
School District is high. Longitudinal attrition figures are prcseatexi in Table 1 
shoving the nuabers of continuing progrsa participants over the first four years 
of the bilingual program. ?or example, of the 290 kindergarten pupils ^rolled 
in the bilingual prograa in 1969-70, 153 continued in the prograa for two years, 
99 for three years, and 75 for four years. 'Becaxise of attrition, the evaluatoV 
felt cccpelled to require* significance at the .01 level. (These figures are taken 
froa ;he application for continuation for the fifth year, Numbers are not extended 
or reported in the 1973-74 fifth year prograa evaluation report.) 

Table 1 

Biling\ial Progras Enrollacnt by Year and by Grade 
Showing Second, Tnird, and Fourth Year Continuing Pupils 



Grade 1969-70 1970-71 1971-72 1972-73 




a — Second Vear Participants; b — Third Year Participants; Fourth Year Participants 
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Third, in 1972-73 a sigaif icaot^ nusbcr of pupils enrolled in an ESL Project 21 
vcrc discovered to ^Pin the control group and were then consequently eldainated 
£ro8 the analysis, gthis in effect reaoved the Spanlsh-doslnant aegsient of the 
control group.) ' 

* Tests vsrc adaialattred by eleaentary coordinators vith the asais- 
tance of classrooe tsachsrs. 

• Prstest and postteat results are reported for both progras partici- 
pants and for control groups vith cosplete and non-tero* scores. 

Results . Table 2 shovs the aean pre- axkl posttest Inter-Aaerican Series Test 
scores for the experlnental and control groups in each -^rode for each year. General 
English ability and sathesatics scores are reported. Vith the exception of grades 1 
and 3 In the 1973-7^ school year, all contrasts significantly favor the bilingual 
participants. Thus the evidence strongly supports the conclusion that the effects 
of the bilingual prograa are positive across grades for the thxee-year period 
reported. The taean gain, differences favoring the prograa participants tend also 
to he large in relation to the estiaated standard deviations bf the gains (esti- , 
aated as the square root o'f tKe within-subject variability across groups). 

Table 3 reports pre-post and gain scores on ths Intsr-Aaerican Series Oeneral 
Ability Test, Spanish Version, for the prograa participants. Ko coaparisons with 
controls are cade. The data indicate that gtrlzis froQ October to Kay are large 
across grades and years. * , - 



Tables 2 and 3 Are ahovn on the follovlng pages. 

Mean Inter-Aaerican General Ability Test scores for grade groups within eac^ 
year were categorized either as control or by length of encollaent in the bilingual 
prograa. In aoat cases, the groups differed significantly although there is no 
apparent positive correlation betveen length of enrollaent and aean score. Posf* 
hoc contraatS'were not calculated. 

Suaaary . On the basis of the data reported for three consecutive yeara, the 
bilingual education prograa in Houston results in significantly higher aeven^aonth 
gains in the English version for prograa participants in cocpariaon to that of 
controls. 

The design applied to suooarize the data is slaple, to the point, and Is one 
that was planned as part of the aanageaent oonitoring process^. It is a suaaative ' 
procedure that could well be copied by aany bilingual prograas that do not stress - 
the sucaative evaluation question vith either clarity or understanding, and con- 
sequently do not perait any coaparative aasessaent to be aad^. 
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Table 3 

Pre- Post and G^in Scores for Experiaeatal Groups 
on the Inter-Aaerican Test Spanish Version 



Grade 


Group 


K 


Pre 


Post 


Gain 


SD 


- F 


P 


^971 


-72 






r 














111 

111 




CO o 


Zo.o 


8.1 


595.3 


.001 , 




^A^c rxoeu wax 


1^ 


AT 7 


37.0 


11.3 


0.3 


194 .1 


.001 ' 


2 


Sxperiaental 




,****. 3 


30 . 3 


1*4 .u 


0.0 


4^4 .0 


.Oul 


- 3 


Bxperi Dental 


79 


60.7 


76.1 


15.4 


6.6 


216.5 


.001 




f 
















K ' 






jO ■ c 


C^ ft 


Z4 .0 


1 ^ ft 


287. 7 


.001 


1 


HxperiacQtal 


1A6 


LI fi " 


30 . X 


lU . 3 


0.7 


104 . 5 


.001 


2 

V 


Experioantal 


161 


•♦J ■ 7 


^0 7 
30 t / 


1^ .0 


7 7 


241 . 0 


.001 


3 


Expe rioental 


218 


*♦ / • D 


01 « 0 


1 /. 7 

14 . c 


11.2 


172.6 


.001 




M Apcrxoen wax 


Ofi 


57. 1 


77.3 


20.2 


13.2 


115.1 


.001 


1973- 


















K 


Experimental 


127 


A6.2 


65.7 




6.2 


605.9 


\001 


1 


Hxperinental 


168 


^3.2 


58.0 


14:8 


9.7 


196.6 


.ocri 


2 


Experinental 


172 


46.1 


62.8 


16.7 


13.9 


12.9 


.001 


3 


Zxperiaental 


163 


54.7 


69.5 


14.8 


12.8 


U.7 


.001 


U 


Experioental 


162 


54.1 


72.9 


18.8 


14.2 


137.4 


.001 
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PROGRAM AREAt 



Ungual Education (Spanish/English) 



PROJECT TITLE; 



Kingsvllle Bilingiul Education Prograa 



LOCATIOX: 



Kingsville, Texas 
SOURCES AKD LEVEL OF PUNDIKG ; 

The following figures pertain to the fourth year of progran operation. 
Title VII Funds ^ 67,727 



PROGRAM START DATE : 
1970 

BRIEF DESCRIPTION OF PROJECT : 

Goals and objectives * Goals are to enable non-English-speaking children 
to achieve proficiency in their oother tongue (Spanish) and in a second lan- 
guage (English), which will penait then to naster the school curriculua ^n 
both languages, and to cultivate pride in their native language and ciilture. 
By achieving these goals, it is hoped that project students will reach grade 
level in all subject areas and that students who are bilingtiAj. will, be able 
to make contributions to both cultures. . » ' 

Context . Kingsville, in Kleberg Councy, Texas, is about 36 iail^s -south- 
w^st of Corpua Christi and 120 niles east of the border povn of Laredo. The 
population is 28,000 and is sooewhat oobile due to a naval base located in 
the city. Attrition from the program, hovever, has not been a problea. 
A celanese plant just outside toyn enploys i^any people; other eoploynent for 
residents is with the city, in services, and with loqal businesses. Kirig^llle 
also the home of Texas A & I University. 

# 

There are.6,505 students enrolled in the Kingsville Independent School 
District, many of whom come from homes in which Spanish is the dominant lan- 
guage. Th« ethnic breakdown in the district is Mexican-American, 32 
black and A3% other. At the elementary level (K-5), about SOX of enrolled 
children are non-English dominant. The one school that was the nucleus of 
the program for four years was almost totally Mexican American. 



Local Funds 
State Funds 



145,000 
990 



Total 



$213,717 



9 
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Prograa oes'cription . ' 

Major features of th'e progran.are the following: ^ 

• use of behavioral 6bjectives and curriculum guides, 

• paired teaching m grades 1-5, [ 

• esphasis on language developatnt in both languages, 
a syftesacic asaessoent of oral language abilities^ 

• pretervice and inaervice training, and 
a parent and cocsunity involves^nt. 

These features will be discussed in appropriate sections below. 

Grade level (s), years of operation, size . The progran is now 
in Its fifth y«ar of operation. In Year 4, the progran served 383 children 
m grades K-A in one school. This year, in order to desegregate,. the district 
clustered two grades in each eleoentary school". There are now three schools 
involved iar the Title VII bilingual education progran. These schools con- 
stitute one of three elenentary level clusters. Kindergarten and first grades 
are "in one school; second and third grades are in another; and fourth and 
fifth grades are in the thirdj school. In these three schools, there are 
now alDost 500 children mvolWd in the bilingual education progran in 
grades In addition, one parochial school is providing bilingual instruc- 

tion for approxinately 70 children. 

Staffing- Last year, full-Cioe staff njcnbers were the director, 
14 teachers, 9 teacher aides, and a secretary. There was also a bookkeeper 
who worked half-tine on the project. Sone classroons also have student 
teachers" froa Texas A & T University. The evAluator and an assistant werfe 
subcontracted to manage the testing "and analysis of data. The assistant is 
also responsible for adnlt^ls taring taped norphologlcal and phonological 
tests to progran participants. Several other people at the district level 
assist the staff of the bilingual progran when necessary. Tnese people are 
the CurriculuQ Director, . Elenentary Supervisor, Counselor, and Visititjg 
Teacher, who visits 'parents at hone. 

Cursicula, tine involved, facilities . 

Behavioral objectives — These have been developed for each 
grade level in the progran and are contained in curriculum guides.. The 
guides suggest procedures and materials' that can be used to teach the objec- 
tives and specify the language(s) in which objectives are to be taught. 
Content areas taught bilingually are the following: language arts in Spanish 
and English, socJial studies (including health, safety,, and culture), and 
mathematics. At the kindergarten level, there are additlqnal objectives 
relating to school orientation (i.e. awareness of school facilities, respect 
for school personnel r respect'^or equipment). The director spot checks teachers 
during the year to, make sure they are teaching to objectives. Teachers nay 
make slight adjustments in objectives to fit needs of the children. 

When the guides were written it was decided that certain vocabulary 
words in Spanish would be used while teaching, especially when referring to 
elements of the home and school environments. This was done to minimize 
cbnfusion for teachers and to lay the foundation for a good standard Spanish 
vocabulary. The procedure was also used to eliminate unfamiliar language 
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found in soae of the readers that caae froa Mexico., If a word was used that 
was Toreign to the children's experience,' teachers cro8,sed it out and sub- 
•^tituted the word that had been agreed upon as being core relevant. This 
provides consistency across the prograa while alsovdeveloping the children's 
vocabulary in' Spanish. / 

- >^ % 

The objectivea in Spanish language arts provide. for reading experiences 
that will enable atudenta to develop decoding skills necessary for the 
effective use of reading for learning. English language area objectives 
are deaigned to teach Spanish-dooinant children proper usage of the Engliah 
language. Objectives for nath and social atudi^es relate cloaely to the 
content of the textbooka used throughout the district. 

Kindergarten — All kindergarten children take the 'Inter-Aaerican - ' 
GeneraUAbility Test in Spanish and English. With parents' pemission, those' 
who dcaonsCrate a language handicap scoring below a certain percentage are 
enrolled in the bilingual education prograa. The core of the kindergarten 
curriculua is a locally developed book containing 32 lessons, each of which 
teaches one sound in the Spanish language. Each lesson includes a high 
interest, action oriented story about the sound and a variety of reinforceaent 
activities. Stories were adapted froa books froa Mexico. The grapheae is 
introduced siaultaneously. Lessons also contain questions about ^the story, 
vocabulary worda using^the sound, and a rhyne. To reinforce the sound, 
children aay be askecf^^to think of other words beginning with that sound, or 
to write the letter on the board*, or to find the letter on a chart. Work- 
books introduce the letter in cursive form, although children are not foraally 
taught aanuscript writing until grade 2. Teachera generally devote one week 
to each lesaon, but there. flexibility to accooodate different learning 
rates of children. ^Students than learn how to coabine lettera to aake syllables 
and syllablea to aake worda, so that by grade 1, ooat children have acquired 
the skills for reading. ' 

Kindergarten classes are s'elf-contained, and all kindergarten teachers 
are bilingual. Inafcructional tiae at all grade .levels is equal for both 
lanj?uages (50Z Spanish and 50Z English). However, kindergarten children 
with very liaiteji abilities in English aay be taught priaarily in Spanish 
until their English language developaent ia sufficient to cope with coapletely 
bilingutl instruction.^ 

Grades 1-5 — Reading instruction in Spanish begins in grade 1 
and -continues to expind upon skills taught in the kindergarten prograa. Once 
the "Child has aastered the skills and begins to read, the focus shifts to 
dev*eloping fluency and coaprehension. .Instruction in English language ^rts 
in grade a continues to upgrade students' verbal abilities, expand vocabularies, 
and enhance understanding of the sounds and structure of Engliah. Children who 
deaonstrate sufficient reaairtess on the Inter-Anerlcan reading readiness test begft 
reading In English during the second semester of grade 1. For those whose English 
speaking vocabulary needs additional development, reading In English is prolonged 
until tn*^ ahow readiness. By the end of grade 2, all children are reading in 
Spanlah and Engliah. Reading instruction in English adheraa cloaely to .the reading 
aariaa usad in the dlatrlct. The currlculua in aoclal atudlea and aath is also 
centered around the texts used district-wide. Achleveaent in thaae areas is measured 
by ceats provided with the textbooks. 
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Itt grade* 1-5, classes are iotegrated so that only, pair t of each « 
class IS involved in bilingual education. Students'oove back andjforth ^ 
between classrooms. Instruction ' in content areas continues to be eqi^lly 
divided betveen Spanish and English, but scheduling is ^uch that student^^in 
the bilingual prpgras are taught in^Spanish by one teacher in English 
by another. Teachers are paired at each gradjt level, and not? all teachers V 
' at the eleaent^ry .level are hilinS^l. Paired taachers plan together to ensure 
chat inatruction is coordinated and ^tuftlly reinforcing. For instance, if 
the bllingtial teacher is teaching a unit in $paniahj>n plant reproduction, 
the other teacher vill esphasize the vocabula't^ add conc^ta during 
language arta ibatruction in English. ' 

Oral language instruction .(K~ 3) — To develop facility in both 
languages, oral langxiage- drills were developed that teach and reinforce the 
structure, sounds, and^yntax of Englaah'and Spanish.* They are^ integral 
part of the curriculua of ali grades and build on sk^ills acquired ^ each 
preceeding ^rade. Drills are practiced aloud-by s'tudencs for thr^^e xo five 
Dinutes each day. Teachers vary the procedures to' avoid boredoa*^ Accoapany-^-* 
ing drills' 'are a aeries of oorphological and phonological teets that are 
adninistered on a pre^est/posttest basis* The 'teats a^asure students* abilities ^ 
CO aake grama tical changes <i.e* knowing plurals for irregular nouns or * ^ " 
changing verb censes)ito speak with inconation and stress, ta'pronounce vovel 
sounds, etc* Tests also provide for a review of corxaonly disused speaking 
patterns often used by Spanish-'dooinant children and for vocabulary expansion. , 
Each child's pretesting session is taped^ and weak areas are tallied. The 
results are returned to^ the teacher, ,who 9ti;ucture8' nuch of *hia or her instruc- 
tion in language developoent. around these identified Areakneases. Posttesting 
occurs. when teachers feel that students heve siastered the drills. 

CI as ar OOP managenent techniques — A pupil profile chart is 
aaii\tained on each child "in, the bilingual prograoS. It ia a cumulative reco];d 
of his- pretest and potttest scores on all standardized tfesta, on casts that 
accompnhy textbooks, and on locally developed language tests* Grouping 
patterns are determined by teachers and are a' function either of*^degree of 
language prof^Lciency or achieveaent level* Within a grade level, groups 
have different coDpoaitions for each subject area, and they change a^ often 
as children's needs change. 

^ '^^ 

Facilities — The j^rograo operates in regula^r^classrooms . No 
mod iftCac ions of school facilic\i?s are required. , ' ^ 

Macerialg- . A smamary'of Che key macerialff used ac all grade levels follows. 

In addicion co che locally developed, scorybook used ^in kindergarcen, 
reading readiness skills are'caughc wich* che Language SkilirTeScc 
from Che Key co Reading Series by che Economy Company, che Learning 
CO Think Series by Science Research Associaces, Alpha Time yith che 
Huggables by New Dimensions in Educacion, and a large aeleccion of 
audiovisual equipmenc. 



J 



D-90 



BoughcoR Mifflin*s Preptrtodgse Para Leer ts the core of the curriculua 
for reidang micrOc^ion in Spanish in gr^e 1^ Other readers uied are 
His Priaeras Letras , Felicidad , and Victoria , all published in Mexico. 
Ejercicios de'Lengnaje and Hx Prib^a Grac^ioa , alao froa Kexico, 
tuppleoent language arts instruction in Spanish. El Kuevo Sesbradori 
a reading series published«in Spain, is the Spanish reading progr^ 
used in grades 2-5. One of the boolta ia al-sp used- in grade 1. ' 
Suppltaentary Sp^ish reaA^rs of varying degrees ^f difficulty, -library 
< books, aad audiovisual aids are alto utilized. Kasy of the library 
^ books are froa' Mexico- ^ . 

Reading m English is taught in alSr grades vith t;be district-adopted 
Houghton Mifflin Reading Prograa. There is cl^se ooniCoring at the 
district level of all students* reading achieveaent in English. 

^ In grades K-3, aatheaatics is taught in Spanish and English vith 
^the Addison-Wesley series, Eleaentary School Mitheaatics, and in 
« g^dea 4-5 with Mo'dem School Mathcaatics froa Boughton Mifflin. 

Social studies is taught with Texto de Unidades Did^ctieas (Diploaa) 
froa Spain and Lands of Proaise published by MacHillan. 

The science curri^Culua uses La. Ciencia ea So Vida froa t)*.C. Keath and 
Coapany and Science by Litidlov, a division of 2>oubleday & Coopany. 

V 

Preservice/lnservict training . For three wssks each suraar, teachers 
attjond a serien of prtssrvice training vorlcahops. They ate scheduled by t"hs 
director and are bailed on particular ne€ds of both teachsrs and studsnts. 
Consultants in aaay aspects of bilingual education have spoken on the follovi^g 
topics: linguistics, teaching aethods in Spanish for speakers of English and 
Spanish, use 'of aaterials and equipaent, and tear^teachiog oethodology. Teachers 
have presented aini-Xessons deaonstrating different techniques. They also have 
sn opportunity to prepare and v*develop cAterials for the coaing school year.' 

-During the year, aonthly inservice training aeetings are held for all 
oenbers of the bilingual staff. As v^th the'suaeer workshops, topics are de- 
teramed by Specific needs of teachers and students. These aeetings have 
been used to further orient new teachers to the project and its objectives and 
to provide aore experienced teachers an opportunity to plan additional class- 
rooa activities- Policy changes within the prograa or at the district level 
are also^discussed. . Teachers evaluate the usefulness of all training sessions 
and aake suggestions of other subjects they would like to pursue in their 
training. 

A third eleaen^ of the staff developntnt cospontnt is vldso-taping of 
teachers *Vhile they teach- This ia done .three tiass a ysar by the director. 
As she observes, she records the types of activities in progress, student 
reactions to the work, ^nd whether the 'activities relate to prograa objectives 
on a teacher ef fectivenes*s analysis fora. She also notes physical characteristics 
of the clasarooa, such as neatness and displays. She then reviews the video- 
tape with each teacher, praising effective techniques and suggesting alternate 
approaches for less* effective aethods. She also points out incorrect langiiage ^ 
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ua-gto- Vith the teachers' permssion, the director has sbova soae of these 
t<i»e» at wor;i»hops conducted for other bilingual -educatios prograiis. Teachers 
«r._uor..ng coward certification la bUmgua: education, soon to be require=ent 
in .cxa». Certification involves attending a 30-hour vorksho? given by one of 
6^^'.Vrtl/t^'°'^ ""^'^ centers, at least one year teaching eicperieace. and 
e"^ .LM"'""''' £ngUsb,spSking teachers 

t .on • K f prograa. this entails receiving 2O0 hours' instruc- 

tion .n Spanish and passing 'a language proficiency test. 

,t-ve7 7^7r^ ?°7r^^?^. ^y^^l^^ Participation b y parents and cogauait v 



!iS»vi-lie'< \>ilLivg\s£t- e-ducation p?^g?£rand parent-co=unicy 
lrllT.^l always been regarjJed as prerequisites for 

lltT.Z,T^"/- "-"^^ f'"'"" °- P""' involvement co=ponent are an 
advisory board. ^ special pirent questionnaire, a parent educTiion progra^ 
and publicising the prograa. . '^^ * • • 

i.«J^o„'^T"°'^ 1°^"^^ 3iUng:ual Prograa. organized the {.rograa's 

.^rf h' 1 °^ • of 

^»!n T° '"5 "P«»<V»"tive8 fro= co=unity organizations and 

agencies, and personnel fro= the district. Parent, not on the Board, teachers. 
wWhT^r. t"/^'°/' "'^""^ generally also attend =;eting,. 

Tit "^'^"1'"^," vhen the =ost people can attend. This is ulullly 

at noonr. Agendas of the aeetmgs include an explanation of the progras's • 
beginnings, its philosophy and goals, deaoastrations of classrocs activities 
planning for the follovfng year, and reviewing proposals. 'The Board also ' 
solicits suggestions from parents about ways to i=prove the prograo. This year, 
because of district^ide clustering of the schools to achieve desegregation 
-he ndvisory Board and Title Vil staff has co=unicated the prograa' s philo- - 
sophy M Che two schools that recently becaae involved. 

A parent questionnaire is .distributed at the beginning of each year to 
T/VJ fn""". willingness and availability to participate in activities such 
as helping with parties. . chaparoning or driving for field trips, te'liag 
ll°ltr' f .^l^J'^ooa. or being a parent representative to organizations 
or out-of-town neetmgs. Farents are' encouraged" to volunteer for at least one 

• Vnf.Zll' f " ^^"^"^ bilingual ciasies and are kept ■ 

mforaed of special activities by notes seat hone with their eliildren. 

Parent education haa been another vehicle for assuring confidence in the 
prograa and enthusiasa for learning. Title ,VlT has sponsored workshops for 
parents on drug education, liealth education, and sex education. Parents are 
in.oraed of other -services and opportunities available to thea through the 
V district, such. 83 b*sic adult education classes. Parents have attended work- 
shops m other cosaunities on bilingual education. 

Inforsatloa about the project i« ditaeninAted tc" the corssunity through 
newspaper articles. -The project also tries to present one or two prograas 
for the cocaunxty each year that jitreas tlie culture and traditions of Mexican 
As^ericans. 
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Costs. Budgets asd enrollments for the firiffour years of program 
operation were as follovs: 



Earollsent Total Budget Fer-pupil cost * 

Year 1 100 $ 97 •800 $978 

Year 2 205 148.907 726 . 

Year 3 293 172.169 588 

Ye^r 4 383 213.717 558 

For Years 1-3. the ?er-pupil cost is cosputed on t^e cocbined total of Title VII 
and local funds. including teacher salaries. In Year 4, the progran began re- 
ceiving state funds, which are included in the per-pupil cost for that year. This 
figure is the araounc expended for the total educational prograa for ^ch child in 
the bilingual education prograsv The peT-pupil cost for Che regular district 
progra= in Kingsville was S835 in 1972-'73 (Year 3 of the bilingual prograa) and 
S954 in 1973-74 (Year 4) and includes all expenditures except debt services and 
capital outlay for equipaent with a lif^ expectancy of oore than one year. 

AccordiUg>to prograa staff, the reason for the drop in per-pupil cost each year 
Is becaus^ the prograa was cpatained In one school'for four years, and materials 
were atcusulated and shared In that school. For exasple. acf^t of the rajor 
Instructional materials purchased in Year 1 for grades K and 1 were still being 
used in 1973-74. but were shared by aore children. The saao' ^cttern held as the ' 
prograa expanded vertically to grade 4 by 1973-74. 

Major start-up coses incurred it Year 1 were $5,265 for contracted services, 
including the auditor, the.evaluator ^ and consultants; $5,160 for classrooa 
and office furniture, audiovisual equipaent. and nonconsuaable Instructional 
aaterials' (textbooics, library books, etc,);^ ^940 for consumable liaterials; 
and $2,540 in travel expenses 5or the adalnlstrative. teaching, and consulting 
staffs. The prograa also received a one-tlae planning grant for $5,167. 
Contracted services ^to^tinued to be a significant expenditure ani averaged » 
about $7,450 for Years 2. 3. and 4. Average travel expenses for the past 
three years increased' slightly to $3.106. Expenses for in^cructiobal equip- • 
sent and cater ials has increase^i as the prograa hafr Increased its enrollment. 
Costs for furniture, audiovisual equi;»ent .' and nonconsunable Instructional 
aaterials have averaged around $10,000 and about $3,300 for consuaable nateria"!^ 
and supplies. 



Overvlev . The Title VII Blilngual Education Progras in King&vi.ie, 
Texts it tn its fif-th year of operation. Tne total co=ruaicy is character- 
ized by lov socio-econoiric conditions. Tnerefore, school achievescat tends 
to be belov tvertge tc all ^rtde le«rels. 

Tne evidence cf progras ef rectiveness reported in this section Is basec 
-^pcn.-daLt-gtthe'-ed for the 1973-7i school year. Bilingual classes in kinder- 
garten through grtde four that year were located in Colston ElcaenXary School 
Control classes tt the sare grade levels vere located tn Kleberg* Hlcacncary 
Scnool. Achieveaent teat scans for bilingual and for control classrooms la 
itlndergtrten through the fifth graae are compared. 

Evaluator. The evaluator of the 'Kingsville bilingual progran is Dr. 
Hariya Barlov, Professor of Education, Texas A and I University, Kingsvilie, 
Texas. 

Pupils . Table 1 snovs tae nurber o: classes anc the n.=ber of pupils 
for vhos pre- ana post-test scores were available on DOth the Spanish and 
!:,ngli8h versions of the Inter-Aserican Series. 



1973-7i Bilingual and Control Classes and Nuaber of Pupils 



N'uaber of Classes Suaber of Pupils 



Group 


K 




2 


3 


U 


Total 


K 


1 


2 


3 


4 ♦ 


Total 


Bilingual 


3 


U 


3 


3 


3 


16 




71 


65 


11 


55 


316 


Control 


2 


2 


2 


2 


2 


10 


41 


17 


27 


28 


33 


141 



During the etrly history o^ the prograst, tttespct to tstigr. cltttct In 
ColHlon to a control group f tiled bectute progrtn sactritlt would be shtred 
i>y tne bilingual classrooa tetchers at Colston. Consequently control classes 
jrere assigned at Kleberg Elenentary School,, a school that aatchec Colston in 
tenas of socio-econoaic level and ethnic aix. Both schools contained approxi- 
sacciy 97-98 percent Spanish-surnaaed pupils. 

Attrition is very lov yitnin the bilingual piograa group. The cocaunity 
is staole, so tnat there is over a 90 percent pupil carryover fron year to year. 

Measures . The f'olloving tests vere adzsinistered to the pupils in both 
tne program and control groups. 

• The SRA Priaary Mental Abilities Test was administered only 
In the first and third grade bilingual classroos In the fall 
of 1973. Scores obtained vere verbtlVceaning , space relations. 



*Durlng the first four yeaps, the prograa vat Inpledpnted only -in the Colston 
School. 
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number facility, perception, total score. 

Both Spanish and English versions of The Inter-Ameriran 
Series: Tests of General Ability were administered to • 
bilingual and control pupils in September 1973 and April 
1974. Classroom raw score means are reported and all possible t- 
test comparisions between classroom means were made. 

• The SRA Achievement Series was administered in April 197A 
to bilingual and program pupils in the first, second and 
third grades. The intention was to predict SRA Achievement 
Series spring scores from the best-weighted lineax combina- 
tion of SRA PMA score and Inter-American Series score obtained 
the previous fail. These predictions were used for defining 
individu4l problem areas so thac special attention could be 
given to pupils with law predicted scores in certain academic 
achievement areas. » ^ 



Methods of comparison . Each classroom mean was compared with other classroom 
means within a grade level. The significance of the mean difference betweea pairs 
of classrooms was based upon a t-test statistic. For the SRA Primary Ability Test 
comparisons, mean differences were tested between all. pairs of the four first grade 
bilingual classrooms and between all pairs of the three third grade bilingual class 
rooms • For each of the five VHA scores, there are six possible contrasts in the 
third grade and 10 possible contrasts in ^^the first grade. Significant differences 
among the classroom groups were revealed at both grade levels. 

The major comparison was based on mean gain scores on the Inter-American 
Series. At each grade level, all possible classroom mean gain contrasts in- 
volving a bilingual and a control classroom are tested by means of t-tests. 
Thus, there is no overall test of total bilingual versus total control group 
mean gain at a grade level, but instead, w^at is reported is the number of 
"possible contrastS'-ii^volvlng a bilingual and a control classroom and the number 
cff these that significantly favor either the bilingual or control classroom. 
_ - ♦ ^ 

No data in the SRA Achievement Tests are reported* 

Results > Mean classroom comparisons on the SRA-PMA all involved bilingual 
pupils, and are not tabled in this section. It is important, however, to not^ 
that there were significant mean classroom differences on all of the PMA tests*-. 

Table, 2 classifies all possible mean gain comparisons involving a control 
and a bilingual classroom at each grade level according to whether the cpmpari- 
son was significant, o^ not and whether the mean differences favored either the 
bilingual or the control classroom* 

<< 

Of the 70 passible bilingual-control classroom comparisons across all 
five grade levels, 25 showed significant mean gain differences on the Inter- 
American Series^ in favor of the bilingual classroom, and five showed mean gain 
significance in favor of the control classroom. ' 

'it ' _ 

On the Spanish version of the Inter-American Series, 24 mean gain compar- 
isons favored the bilingual group and 17 of these were significant at the 
alpha * .05 level. Eleven comparisons favored the control group and none of 
these were significant. Thus in terms of the acquisitipn of Spanish language 
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TABLE 2 

Classroon TrTest Cocparison Survey: 1973-74 Kingsville 







K 


1 






2 


3 




4 




Total 


Results 






B 


C 


B 


C 


B 


C 


B 


C 


B C 


English: 


















> 






Significant^ 


6 


0 


1 


2 


0 


0 


0 


3 


1 


0 


'8 5 


Kot Significant 


0 


0 


3 


2 




2 


2 


1 


5 


■3 


U 8 


TOTAL 


6 


0 








2 


2 




6 


3 


22 13 


Spanish: 












— 












Significant 


6 


0 


2 


0 




0 


2 


0 


3 


0 


17 0 


Not Significant. 


0 


0 


2 




1 


1 


1 ' 


3 


3 


3 


7 11 


TOTAL 


6 


0 . 


4 


4 


5 


1 


3 


3 


6 


3-1 


24 11 



« Bilingual mean higher t/ian control 

» Control mean high than bilingual 

= Significant at the alpha « .05 level 



skills^ the bilingual classes outgained the control classes at all grade levels. 
Nearly half (17 out of 35)J possible bilfngual-control contrasts were significant 
and all favored the bilingual classroo%. ^ * 



On the English version of the Inter-American Series, there were 13 signif- 
icant dnd 23 insignificant. mean gain differences between bilingual and control 
classes. Of the 13 significant contrasts, eight favored the bilingual class 
and five favored the'' control; six of the eight comparisons favoring the bilingual 
class were at the kindergarten level; two of the seven significant conttasts for 
grades 1 through 4 favored the bilingual classes. While the evidence is npt as 
strongly favorable on the English as on the Spanish versions of the test, the* 
bilingual classes nevertheless ^did outgain the control classes in kindergarten. 
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bILINGv'AL PROGRA.M DEStRIPTIOH OUTLINE 



Pr ograa Overvlev (500-800 word summary) 



Identification information 



t program title 

• location name 

• year started 



Background information 

• historical development 

• context 

• needs assessment 

Objectives and procedures 

• bilingual/bicultural philosophy 

• ^^major objectives and raciorrale 
'primary program features and rationale 

• key instructional strategies 

• time devoted to instruction in both languages 

• use of language in content area instruction 

Participants 

• age, grade level, sex, number 



^ demographic^ackground 

• language dominance 

• special characteristics * 

• language baseline competencies 

• selection criteria (qualifications) 

• selection procedures (test, referral, screening panel) 
Personnel 

• categories arid number 

» qualifications and training ^ . . 

« 

Program Development 

Relation' of program to t^*rgat population and school system 

• target populat^j^n sM2e -::aiid' composition - 

• demographic and cultural actors 

• geographical area served, physical 's^ize> number of schools by grade 

• integration of program in system and with other programs 

History and r.eeds assessment 

• origins and philosophy 

• needs assessment focus methods, and priority selection 

• impact on program development 

Bilingu^ai/blcuLciuaJ ^ pro^.i c.m objectives and rationale 

• objecllve^ witn rcspect i) ianguage^dominant subgroups 

• general strategy for measuring attainment of objectives 
exper-ted extremes 
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Planning the program (prior to start) 

• key persons/roles in major planning tasks 

• major -steps, e.g. 

enlisting support 
identifying resources 
J obtaining funds 

developing/adapting curriculum materials 
assembling a;id training staff 
s'electing irarticipants 

Changes /growth in original program! 

• pattern of growth and development 

• major changes, if any, and rationales for them 

• major problente, if any, and strategies for solving them 

Staffinj^ and Management 

Staff types, numbers, and chain of command 

• job descriptions and roles 

• time devoted to program, length of service in program 

• qualifications, certification, experience in bilingual education 

• competencies in second languages 

Rectuitment of staff 

• sources 

• bilingualism requirements and experience 

• selection strategies 

• strategies for maintaining a stable, motivated staff 

Preservice and inservice training 

• objectives 

• relationship to wider bilingual/bicultural program objectives- 

• key persons/roles in conducting training 

• extent, schedule, activities, methods of training 
. • assessment of training program 

Management strategies 

• how. chain of command is implemented on daily basis 

« program administrator's autonomy and authority at -district and 
school levels 

• key management strategies, and rationales • — = == 

IV. Instruction 

Language of instruction 

• choice 

• rationale 

• extent of use in instruction ^> ^ 

• attitude of teachers and children to language oi instruction 

1 - ^ 

Writers were free to discuss changes the program had undergone in- - 
each chapter, as appropriate. 
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Instructional activities by grade level and by language/content areas 

• relation of methodology to objectives and philosophy of program 

• bilingual teaching techniques 

• other teaching techniques 

• cultural component of th~e curriculum 

• motivational/reward techniques 

• feedback to students and parents - * ■ 

• classroom management 

• classroom climate 

Grouping 

• objectives and purposes 

• criteria and patterns 

• frequency of grouping children for daily instruction 

• frequency of regrouping 

Diagnosis and assessment of participants 

• specific competencies measured 

• measurement procedures, instruments, frequency, records, reporting 

• use of diagnostic/assessment records for decisions 

• other observational procedures 

Typical schedule/timeline' by grade level 

• time devoted to dominant and second language instruction in 
each subject 

• integration of language subgroups for scheduled activities 

Physical layout of instructional £acilities 

• features of plant that affect instruction 

• features of classroom that affect instruction 

Key materials and equipment 

• list of key items and quantity required 

• procedures/rationales for selection, adaptation, development, and use 

• cultural reference/language of key items of materials 

V. Parent V Involvement and Community Awareness 

Rationale and purposes 

• relation to program philosophy and objectives 

• intended outcomes 

Historical perspective ^ ^ ^ 

• trends in participation 
f changes in scope or emphasis 

Specific nature 'of involvement 
^ • key groups and their impact on the program 

• school- and class-based activities 

• Jiome-based program activities ' ^ 

♦ • extra-school involvement in program planning and review 

*• ' 

Measures of involvement 4 , 
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Results 



VI. Costs 



Sources and level'.of funding 

• Federal funds 

• State funds i 

• Local funds 

• Other funds 

Analysis of program-supplies' cost information 

• Start-up costs by number of participants^ 

major cost categories 
one-time costs 

^ Continuation costs by number of participants for succeeding years 
major cost categories 
^ " consumable items, reusable items 

cost trends 
* • Per-pupil costs 

basis for computing 

state or local per-pupil allocation to which/added on, 

if applicable 
trend in per-pupil costs 

compared to per-pupil cost of regular program 

Guidance for developing a replication budget * ' 

• Further interpretation and comments re cost data 

• Budget options ^ ^ 

VII. Evaluation 

Design 

Measures 

Results 

Interpretation . ' 

VIII. Sources for Further Information 

Program director ' ..i 

Evaluation director 

Materials and equipment sources , 
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